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blade server in the industry! 





The new Triton TwinBlade Server is the most technologically advanced blade server system in the industry, 
and the ideal solution for power-efficiency, density, and ease of management. 


The Triton TwinBlade Server supports 
up to 120 DP servers with 240 Intel® 
Xeon® 5600/5500 series processors 
per 42U rack, achieving an umatched 
0.35U per DP node. Up to two 4x QDR 
(40 Gbps) Infiniband switches, 10GbE 
switches or pass-through modules give 
the TwinBlade the bandwidth to support 
the most demanding applications. 


With N+1 redundant, high efficiency 
(94%) 2500W power supplies, the 
TwinBlade is the Greenest, most energy- 
efficient blade server in the industry. The 


energy saved by the iX-Triton TwinBlade 
Server will keep the environment cleaner 
and greener, while leaving the green in 
your bank account. 


Server management is also simple 
with the Triton Twin Blade Server. 
Remote access is available through SOL 
(Serial Over Lan), KVM, and KVM over 

IP technologies. A separate controller 
processor allows all of the Triton’s remote 
management and monitoring to function 
regardless of system failures, offering true 
Lights Out Management. 


Using the Triton’s management system, 
administrators can remotely control 
TwinBlades, power supplies, cooling 
fans, and networking switches. Users 
may control the power remotely to 
reboot and reset the Triton TwinBlade 
Center and individual Twin Blades, and 
may also monitor temperatures, power 
status, fan speeds, and voltage. 


For more information on the iX-Triton 
TwinBlade, or to request a quote, visit: 


http://www.iXsystems.com/tritontwinblade 


20 Server Compute Nodes in 7U of Rack Space 


The iX-TB4X2 chassis holds 10 TwinBlade servers and each 
TwinBlade supports two nodes. This gives the iX-TB4X2 chassis the 
ability to house 20 nodes in 7U of rack space. The powerful Triton 
TwinBlade achieves 0.35U per dual-processor node, and is twice as 
dense as the previous generation of dual-processor blades. 


A fully-loaded iX-Triton TwinBlade supports 40 Intel® Xeon® 
5600/5500 series processors and up to 2.5 TB DDR 
1333/1066/800MHz ECC Registered DIMM memory. In a 42U rack 
this translates into 120 nodes with 240 Intel® Xeon® 

5600/5500 series processors and 15 TB DDR 1333/1066/800MHz 
ECC Registered DIMM memory. 


» By replacing 1U servers with TwinBlade servers, the power 
savings of the iX-TB4X2 can reach more than $1000* per 
year, per server with reduced cooling costs added in. 


» Replacing 1U rackmount servers with an iX-TB4X2 Twin 
Blade can reduce carbon dioxide emissions by over 5.5 
metric tons.** 


» The iX-Triton TwinBlade delivers the most energy-efficient 
blade server in the industry with four N+1 redundant, high 
efficiency (94%) 2500W power supplies. 





* Electricity costs vary by location. 


** According to Energy Information Agency (a statistical agency of the U.S. Department of Energy), 
saving one kilowatt hour of electricity reduces carbon dioxide emissions by 1.43 pounds. 





Call iXsystems toll free or visit our website today! 
+1-800-820-BSDi | www.iXsystems.com 


Intel, the Intel logo, and Xeon Inside are trademarks or registered trademarks of Intel Corporation in the U.S. and other countries, 
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Up to 10 dual-node TwinBlades in a 7U 
Chassis, 6 Chassis per 42U rack 


Remotely manage and monitor 
TwinBlades, power supplies, cooling fans, 
and networking switches 

Hardware Health Monitor 

Virtual Media Over Lan (Virtual USB, 
Floppy/CD, and Drive Redirection) 
Integrated IPMI 2.0 w/ remote KVM over 
LAN/IP 

Remote Power Control 

Supports one hot-plug management 
module providing remote KVM and IPMI 
2.0 functionalities 


Up to four N+1 redundant, hot-swap 
2500W power supplies 


Up to 16 cooling fans 


Each of the TwinBlade’s 
two nodes features: 


Intel® Xeon® processor 5600/5500 series, 
with QPI up to 6.4 GT/s 


Intel® 5500 Chipset 


Up to 128GB DDR3 1333/ 1066/ 800MHz 
ECC Registered DIMM / 32GB Unbuffered 
TEU 


Intel® 82576 Dual-Port Gigabit Ethernet 
2 x 2.5" Hot-Plug SATA Drive Trays 
Integrated Matrox G200eW Graphics 


Mellanox ConnectX QDR InfiniBand 
40Gbps or 10GbE support (Optional) 


Powerful. 
Intelligent. 
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Dear Readers! 


The first month of summer is coming to an end, | 
guess most prorably many of you are on vacations 
having good time with your friends and family. |am 
happy you dont forget about BSDMagazine and 
download it every month. :) 


BSD Magazine is growing, it has already around 
22 000 subcribers all over the world. Comparing 
to 10 000 printed copies which-were distributed in 
USA before January - this number has really grown! 
We are looking for the new ways to promote our 
magazine all the time and we are very grateful for 
every help you give us! Thank you for spreading a 
word abour BSD Mag! 


This issue is devoted to OpenBSD mainly, but 
not only. A little bit of firewalling, floppy systems, 
sharing interesting experience and other. All these 
great authors worked hard to contribute to this 
issue: Dru Lavigne, Juraj Sipos, Daniel Gerzo, Daniele 
Mazzocchio, Sasan Montaseri, Joshua Ebarvia, Jesse 
Smith. In case you have any questions about the 
articles, just let us know we will forward them directly 
to authors. 


Enjoy your reading and have a nice day! 


Olga Kartseva 
Editor in Chief 
olga.kartseva@software.com.pl 
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GET STARTED 


OG Building a Desktop Firewall with pf and 
fwbuilder 
Dru Lavigne 
This article is an excerpt from the Firewalls and VPNs chapter of 
the book The Best of FreeBSD Basics (ISBN 9780979034220), 
published by Reed Media Publishing. 
Everyone knows that you should be behind a firewall whenever 
you go online. However, not everyone knows that it’s easy 
to create a personal firewall for a FreeBSD (or PC-BSD or 
DesktopBSD) system. This section shows how even a casual 
home user can get a firewall up and running in about ten 
minutes 


HOW TO’S 


412 OpenBSD Some Interesting 
One Floppy Systems 
Juraj Sipos 
One floppy systems are very practical, as 
they usually have a specific goal, which 
cannot be said about all Live CD’s. 
















16 Remote Installation 
of the FreeBSD 
Operating System 
without a Remote 
Console 
Daniel Gerzo 
This article documents the 
remote installation of the FreeBSD 
operating system when the console 
of the remote system is unavailable. 
The main idea behind this article is the 
result of a collaboration with Martin Matuska 
mm@FreeBSD.org with valuable input provided 
by Pawel Jakub Dawidek pjd@FreeBSD.org. 


oc OpenBSD as Mail Server 

Daniele Mazzocchio 

In a previous document, we built redundant firewalls 
using the CARP and PFSYNC protocols; these were the first 
building blocks of a hypothetical, OpenBSD-based, small private 
networkthat we are going to build step by step across several 
documents. 
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LET’S TALK 


Performance Comparison ITTIA DB and 

SQLite 

Sasan Montaseri 
ITTIA DB SQL and SQLite are used by software developers 
to manage information stored in applications and devices. 
Designed to be hidden from the end-user, these embedded 
relational database management systems are linked into the 
application or irmware as self-contained software libraries. 


Interview with Jeff Roberson 

Jesse Smith 

Any administrator who has rushed to bring a system 
back on-line after a crash knows how frustrating it can be to 
sit through a filesystem check. It can be a painfully slow, yet 
necessary process. One BSD developer, Jeff Roberson, has 
found a way to make all our lives easier and system recovery 
faster. Jeff took some time out of his very busy schedule 
to explain some of the bottlenecks in filesystem 
recovery and how he has gone about 
speeding up the process. 


FreeBSD 50 
Experience and 

Succes Story 

JOSHUA EBARVIA 

In 2007, | was hired as a 


programmer at University of 
the Philippines Open University 
(UPOU). | came from a Microsoft 
Windows platform and Visual Basic 
background. At UPOU, there was no 
room for for my skills since they use 
various distributions of Linux for servers 
and open source programming languages for 
applications development. 
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GET STARTED 


Building a Desktop Firewall 


with pf and fwbuilder 


This article is an excerpt from the Firewalls and VPNs 
chapter of the book The Best of FreeBSD Basics (ISBN 
9780979034220), published by Reed Media Publishing. 





What you will learn... 
« how to build and configure a basic firewall using pf and fwbuil- 
der ona FreeBSD system 


whenever you go online. However, not everyone 

knows that it’s easy to create a personal firewall 
for a FreeBSD (or PC-BSD or DesktopBSD) system. This 
section shows how even a casual home user can get 
a firewall up and running in about ten minutes. 


= veryone knows that you should be behind a firewall 


The Software 
Like all of the BSDs, FreeBSD has always been security 
conscious. It offers several built-in firewalls to choose 
from: ipfw, ipf, and pr. | use pt because it is built into all of 
the BSDs, including OpenBSD, NetBSD, and DragonFly 
BSD. 

| also recommend using a GUI firewall editor called 
fwbuilder. While my examples will demonstrate this utility 
from a FreeBSD system, it is available for Linux, Mac OS 
X, and Windows XP and supports iptables, ips (IP Filter), 
pf and ipfw. pf comes with FreeBSD, but double-check 
that it is loaded on your system by typing the following as 
the superuser: 


# kldload pf 


If you get your prompt back, you just loaded it manually. 
If you’re in the habit of turning off your computer, add 
a line to /etc/rc.cont to reload pr when your system 
boots: 
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What you should know... 
+ basic knowledge of TCP/IP and TCP/UDP ports 


pf_enable="YES" 





© fwbuilder 


Here you can add or edit interfaces manually. ‘Name! corresponds to the name of the 
physical interface, such as ‘eth0', 'fxp0'" 'ethernet0' etc. 'Label' is used to mark 
interface to reflect network topology, e.g. ‘outside' or ‘inside’. Label is mandatory for 
PIX firewall. 


Click 'Next' when done. 


Netmask 











(<[>) 





@ Regular interface 


© Dynamic address 


© Unnumbered interface 




















© Bridge port 
MAC: 
| Add ] | Update ] | Delete 
| < Back | Next > || Finish || Cancel | 

















Figure 1. Your new firewall 
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@ FreeBSD 7.2 Jewel Case CD/DVD 
Set contains: 


* Disc 1: Installation & Live File System (for system recovery) 

+ Dise 2: Packages and Documentation 

+ Dise 3: Additional Packages 

+ Dise 4: More Packages 

FO OEES DD FD ODD iccssccensncinnisnomeoenneininnanan cinasmmenatianiaominitsiancianinnasiasnabta 
FreeBSD 7.2 DVD......... 
FreeBSD 6.4 CDROM .. aces 
FRO ENS ED G25 DVD cass ccoscasiieocccscseccems cess mmemvobonpsaccupoahenesonaceroestaos 





@ FreeBSD Subscriptions 


Save time and $$$ by subscribing to regular updates of FreeBSD! 


FreeBSD Subscription , start with CD 7.2 ..........csssssssesesssseeeeseee $29.95 
FreeBSD Subscription, start with DVD 7.2 .....ccccccsesseseseesereseene $29.95 
FreeBSD SubSCription, CD 6.4 ........scseccssssssssecseesesseseseesessneeeseense $29.95 
FreeBSD Subscription, DVD 6.4 ...ccsssscssssssssesesssssessnsessnnecessnseesave $29.95 


@ PC-BSD 7.1 DVD (Galileo Edition) 


PCBS 7E AIUD sscssscsssssacsnsescncsanssscapsspassncncapaniagcovanmsscapaaseensiosans $29.95 
PC-BSD SUBSEHIMION cacs. e eeee $19.95 


@ BSD Magazine 


BRIE 6ocss ccc etme 
BSD Magazine Subscription 








Your FreeBSD & 
PC-BSD Resource 


www.FreeBSDMall.com 





@ The FreeBSD Handbook 


The FreeBSD Handbook, Volume 1 (User Guide) ....s.sssssseses00e 939.95 
The FreeBSD Handbook, Volume 2 (Admin Guide) ....... 
© Special: The FreeBSD Handbook, Volume 2 (Both Volumes) 
© Special: The FreeBSD Handbook, Both Volumes, & FreeBSD 7.2 ..... $79.95 





@ The FreeBSD Bundle 


Inside the Bundle, you'll find: 


+ FreeBSD Handbook, 3rd Edition, Users Guide 
+ Fre@BSD Handbook, 3rd Edition, Admin Guide 
+ FreeBSD 7.2 4-disc set 

+ FreeBSD Toolkit DVD 


® Special: The FreeBSD CD Bundle ..........s:ccsecesssssorssesrenssssssesessevsnsesones 
© Special: The FreeBSD DVD Bundle 





@ The FreeBSD Toolkit DVD........s39.95 
@ FreeBSD Mousepad................ $10.00 
@ FreeBSD Caps $20.00 
WD PC-BSD Caps 2 ececeennsnnneninn $20.00 


For MIORE FreeBSD & PC-BSD items, visit our website at FreeBSDMall.com! 


t-shirts 
$18-$21.99 


CALL 925.240.6652 Ask about our software bundles! 


If you instead get an error like: 


kldload: can’t load pf: File exists 
it means that your system is already configured 
to load pf for you. 


Installation and Configuring the Firewall 
Object 

From the GUI, become the superuser and install 
and start twouilder: 


GET STARTED 





Listing 1. Using ifconfig to find out your interface names, IP addresses, and 
MAC addresses 


# ifconfig 
x10: flags=8843<UP, BROADCAST, RUNNING, SIMPLEX,MULTICAST> mtu 1500 
options=9<RXCSUM, VLAN MTU> 

inet 192,168.2.49 netmask Oxff£f£EEL00 broadcast 192.168.2.255 
ether 00:04:75:ee:e0:21 
Ethernet autoselect 


media: (100baseTX <full-duplex>) 


status: active 


100: flags=8049<UP, LOOPBACK, RUNNING, MULTICAST> mtu 16384 


# pkg_add -r fwbuilder 
# rehash 
# fwbuilder 


Name: 
This will take you to the fwbuilder GUI, which is | Label: 
divided into two main sections. The left frame | address: 
contains an Object tree and the right frame | Netmask: 


contains your firewall rules (after you have | mac: 
defined some objects). Using objects is a very 


powerful visual aid, allowing you to quickly | Name: 
see your networks, computers, and services, | Label: 
and to cut and paste these objects into firewall | address: 
rules. Netmask: 


The first object you create should represent your | mac: 
firewall. Click on the New Object icon (it looks like 
a sheet of paper) and select New Firewall from 
the drop-down menu. Give your firewall a name (I 





inet 127.0.0.1 netmask 0xff£000000 


Listing 2. Interface information for example firewall object 


x10 

external 
(greyed out because I checked Dynamic address) 
(greyed out because I checked Dynamic address) 
00:04:75:ee:e0:21 


100 
loopback 
LAT Oa aal 
23350 50),0) 


(leave empty) 








called mine my_firewai1), select PF from the drop- 
down menu of firewall software, and click Next. Keep the 
default to Configure interfaces manually, and press Next. 
You should see a screen like Figure 1. 

Be sure to Add the interface information for each NIC 
in your computer as well as the loopback. If your firewall 
will protect only your personal computer, you need only 
one physical NIC installed in your computer. If you wish 
your computer to provide NAT to other computer(s) 
on your home network, you need to have two NICs 
installed. 

If your ISP assigns you a DHCP address, check the 
Dynamic address option. Otherwise, enter your static IP 
address and subnet mask. 

To determine the FreeBSD names of your interfaces as 
well as the associated IP addressing information, type: 
see Listing 1. 

With my information, | entered into the New Firewall 
screen: see Listing 2. 

When choosing a label, external is good for the NIC 
you use to access the internet, and internal is good for 
the NIC attached to your home network. If you need to 
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add a static subnet mask, you must first convert that hex 
number (oxereer£00, for example) to decimal. Ignore the ox, 
as that simply indicates a hex number. What remains is 
four pairs of numbers: «+ tr e+ 00. ff is easy; it represents 
255; and oo represents o. So this mask is: 255.255.255.0. If 
you have a pair that isn’t an ff or a oo, use the conversions 
in Table 1. 

Note to users of modems: your interface name will be 
either pppo Or tuno. Running i¢contig while connected to the 
Internet will make it easier to spot your IP address. 

Once you’ve entered the information for a NIC, 
click Add and repeat for each of your NIC(s). When 
finished, click on the Finish button. If you take a look 
at your Object tree, it now contains some new objects: 
one for your firewall and one for each interface you 
defined. 

You have one last change to finish the firewall object — 
marking one of the interfaces as a Management interface. 
For a personal firewall, it should be the loopback. Double- 
click your loopback object and check the Management 
interface box, then click Apply. 


07/2010 





Building a Desktop Firewall with pf and fwbuilder 


Table 1: Hex Conversion Table 


80 128 
cO 192 
e0 224 
fO 240 
f8 248 
fc 252 
fe 254 


Creating a Simple Firewall Ruleset 

You now have everything you need to create a simple 
firewall ruleset that allows your personal computer to 
access the internet and prevents anyone on the internet 
from accessing your computer. 

Highlight the Policy object under your firewall, then 
click on the Rules menu and select Insert Rule (see 
Figure 2). 

Notice that the default rule denies any source from 
reaching any destination using any TCP/UDP service. 
To allow the system running the firewall, right-click your 
firewall object and select Copy. Right-click inside the 
Source box of the rule and Paste. Your firewall should 
now show as the source of packets. Next, right-click 
the Deny word under Action and change it to Accept. 
In the Options box, right-click and select Logging Off 
— you don’t want to log every one of your successful 
packets. 

You should always add a comment to remind yourself 
why you made a rule. If you double-click on the box, you 
can type in your comment. | wrote: allow my computer to 
access the internet 

That one rule is enough to give you a working firewall. If 
you want, you can add a second rule. With your existing 
rule highlighted, click on the Rules menu and select Add 
Rule Below. Add a comment: deny all other traffic. 

If you don’t plan on looking at your firewall logs, turn off 
logging in the Options box. 

Note that this second rule isn’t necessary for this setup, 
because the ps firewall assumes you want to deny any 
traffic you didn’t explicitly accept. This is known as an 
implicit deny. You may find it useful to add the rule with 
a comment to remind you of this behavior. 

Tip: A quick administrator’s trick is to add this rule only 
when you are troubleshooting a problem and to leave the 
Logging option on. 


Installing your Firewall Rules 


You’ve just created a firewall ruleset, but it won’t start 
working until you install it. 


www.bsdmag.org 


First, you need to configure ssna to allow the superuser to 
connect and install the firewall rules. By default, FreeBSD 
doesn’t allow superuser ssh sessions. Change this default 
by typing the next line very carefully and double-checking 
your upper- and lowercase and your >> before pressing 
enter: 


# echo "PermitRootLogin yes" >> /etc/ssh/sshd_contig 


Don’t worry; no one on the internet will be able to ssn to 
your computer once you install your firewall rules. 
Next, tell sshd about that change: 


# /etc/rc.d/sshd reload 


Reloading sshd config files. 
If you see an error: 


sshd not running? (check /var/run/sshd.pid). 


use this command instead: 
# /etc/rc.d/sshd start 


Starting sshd. 
Double-check that sshd is running with: 


# /etc/rc.d/sshd status 


sshd is running as pid 5467. 

Next, select Install from the Rules menu. Make sure 
both boxes are checked to Compile and Install the firewall. 
Click Next which will prompt you to select a filename and 
location to save your fwbuilder policy (it will end in a .fwb 
extension). In the next men, enter root for the username 
and type in the password for your superuser account. 





: Firewall Builder - (untitied) AA ® 
2 Fie Edt Object ules Tools Wedow Help =|8)= 





my_firewall / Policy 1 














Object Type: Firewat 
Object Name: my_frewat Sr 7 
Piatform: pf 

“ y * Host OS Sett Comment: 
Host OS: freebsd Name: my_frewall ange 

weston Sat jun 5 18:39:47 2010 wiotibrant ltt Frewal settngs 
Inactive firewall | 





instaled: Version: [= any - 


Host OS: FreeBSD 











Figure 2. Inserting a rule 
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Review the options, then press OK. You should receive 
a New RSA key message: 

You are connecting to the firewall my sirewai1 for the first 
time. It has provided you its 

identification in a form of its host public key. The 
fingerprint of the host public key is: 

b6:76:30:aa:01:27:64:48:3b:18:28:18:5b:c9:ae:e4 You can 
save the host key to the local database by pressing YES, 
or you can cancel connection by pressing NO. You should 
press YES only if you are sure you are really connected to 
the firewall my firewall. 

It is safe to press Yes because you know you are 
connecting to your own firewall. However, it is good to 
know how to check a host's fingerprint in case you ever 
connect to a remote FreeBSD system: 


# ssh-keygen -l1 -f /etc/ssh/ssh_host_dsa_key.pub 
1024 
b6:76:30:aa:01:27:64:48:3b:18:28:18:5b:c9:ae:e4 


Note: You will only need to verify the fingerprint the very 
first time you install your firewall. 

Once you click Yes, the policy will be installed and 
should indicate a Progress of Success. Your firewall is 
now running. 

Note: The installation may fail if p< is already running. 
Try running the command péct1 -a to disable the firewall 
and then retry the Install from the Rules menu again. 


Controlling the Firewall 

Use the prcti (pe control) command to see what's 
happening with your firewall and to stop and start the 
firewall. Use the show switch (-s) to view the rules 
currently running on the firewall: 


# pfctl -s rules 

pass out quick inet from (x10) to any flags S/SA keep state 
label "RULE 0 -- ACCEPT " 

block drop quick inet all label "RULE 1 -- DROP " 

block drop quick inet all label "RULE 10000 -- DROP " 


If you compare that text to the rules you made in 
fwbuilder, you'll recognize rules 0 and 1. Rule 10000 is 
that implicit deny rule. 

If you ever wish to stop your firewall, use the disable 
switch: 


# pfctl -d 


To restart the firewall, specify the name of your ruleset. It 
will be in /etc and have the same name as your firewall. 
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In my case, it is in /etc/my _firewall.conf. To start this 
firewall, | use prct1 at the command line to load the rules 
and enable the packet filter: 


# pfctl -e -f£ /etc/my_firewall.conf 


Alternatively, | can right-click the firewall in the Objects 
tree and choose Install from the drop-down menu. 

Note: If you added the line to /etc/rc.cont mentioned at 
the beginning of this section, add another line to load your 
ruleset if you reboot your computer: 


pf_rules="/etc/my_firewall.conf" 


where my _firewall.conf Is the name of your ruleset. It is 
always a good idea to run pfct1 -s rules after a reboot to 
double-check that your firewall is running. 


Conclusion 

I've demonstrated how to make a personal firewall that 
protects your system while allowing you to access the 
internet. The next section will show you how to install 
a NAT policy with ¢wouiider and explore some of its other 
features. 


Original Article: 
http://www.onlamp.com/pub/a/bsd/2006/08/03/FreeBSD 
Basics.html 








DRU LAVIGNE 

Dru Lavigne is a network and systems administrator, IT instruc- 
tor, author and international speaker. She is author of O’Reilly- 
’s FreeBSD Basics column and the books BSD Hacks, The Best of 
FreeBSD Basics and the Definitive Guide to PC-BSD. 

She is currently the Editor-in-Chief of the Open Source Busi- 
ness Resource, a free monthly publication covering open sour- 
ce and the commercialization of open source assets. She is fo- 
under and Chair of the BSD Certification Group Inc., a non-pro- 
fit organization with a mission to create a standard for certify- 
ing BSD system administrators. Dru is a Director at the FreeBSD 
Foundation. 
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OpenBSD 


Some Interesting One Floppy Systems 


One floppy systems are very practical, as they usually have 
a specific goal, which cannot be said about all Live CD's. 


What you will learn... 

+ Readers will learn something about how to use serial console 
in OpenBSD, how embedded systems can be made, or what is a 
transparent firewall 


as router, you must make some configurations; 
with a floppy system already designed to work 
as router no such configuration is needed (except for 
the most basic one like putting the proper names of 
network devices into configuration files). Such a diskette 
is portable and easy to use. You may, too, have 
other wishes — an MP3 player on a diskette, or even 
a transparent firewall. 
| am the author of SONaFR and KarmaBSD —- two 
quite interesting one floppy systems. MaheshaBSD 
is a bit larger project of mine, but | already wrote an 
article about this LiveCD in the May issue of the BSD 
Magazine. SONaFR is a router based on OpenBSD 4.1 
and the latter one (KarmaBSD aka 1FCD-OpBSD) is 
a one floppy MP3 player with a number of possibilities 
| describe later. 


Ty: configure OpenBSD installed on your hard disk 


SONaFR 

There are not many one floppy OpenBSD routers or 
firewalls and my project usually always appears on 
the top in Google with keywords like: one — floppy 
— router — OpenBSD. Another floppy router is fdgw 
(http://www.fmi.org/software/fdgw/), but it is based on 
NetBSD. One of the best one floppy OpenBSD projects 
is foaf (Floppy OpenBSD Firewall) (hitp:/vww.theapt.org/ 
openbsd/firewall. html). 
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What you should know... 

+ how to setup network connections with ifconfig 

+ they should read manual pages of some important OpenBSD 
commands (pfctl, disklabel, fdisk, etc.) 


How to use SONaFR 

This floppy distro has a minimal kernel. You must have 
two network interface cards (NIC's) in a computer where 
you use this floppy. To see all the network cards available 
on your system, type: 

ifconfig (from within SONaFR after it boots). 

To see all the cards that the SONaFR kernel supports, 
type: 


more etc/cards 


The configuration scripts of SONaFR (for example, / 
etc/pf.conf) may be immediately used in any OpenBSD 
hard disk installation for firewall/router purposes; thus 
anybody can learn how to configure the OpenBSD 
packet filter. 


Transparent Firewall 

This thing may also be used as a transparent firewall 
(invisible firewall). If you have a computer with two 
NIC's (the third NIC may be used only for a SSH login 
with purpose to control such a transparent firewall) 
and you move data from one network card to another 
one via a bridge (without IP addresses), you work on 
the OSI layer 2 model (data link); thus, if you move 
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data this way over firewall, the advantage is that you 
may put such a firewall anywhere — you can split any 
network segment without needing to configure anything 
(except for the transparent firewall). Such a firewall 
is very quick, as no decisions need to be made with 
respect to IP addresses a normal firewall always 
requires. Bandwidth, too, may be easily reduced (with 
use of ALTQ —ALTernate Queueing framework for BSD 
UNIX). 

A good (and quick) overview of transparent firewalling 
in OpenBSD with tips and setup requirements can 
be found here: http:/Wwww.dalantech.com/fusionbb/ 
showtopic. php ?tid/7 1026/pid/7 1026/postlast/m/1/. 

First, you must create the bridge (type the following 
command from within SONaFR): 


ifconfig bridge0 create 
Then activate the bridge: 
brconfig bridgeO0 add rl0 add rl2 up 


(replace "rl0" and "rl2" with real network devices present 
on your system.) 

To activate the transparent firewall, you have to run 
the pfctl command (for packet filtering; the /ectc/p#.conf 
file needs to be edited if you have special requirements; 
SONaFR has a little editor for such a purpose, just type: 


mg): 
pfctl -f /etc/pf.conf 


The behavior of such a firewall depends on rules defined 
in the /etc/pf.cone file. 

To run SONaFR, your minimal requirements must be at 
least 9,5 MB of RAM and a Pentium (486 too) computer 
with two network cards and a working diskette drive. 
However, the SONaFR's ability to detect all possible 
network cards (NIC's) is limited. This is because the 
system has a minimalist kernel. 

The /etc/p£.cont file in SONaFR works immediately. If 
you have more requirements, look into pf (Packet Filter) 
Internet tutorials and edit the ps .con¢ file appropriately. The 
purpose of this distribution was to bring something quick 
and easy-to-use. The only thing that the user must do is 
to substitute the nreo and rio entries in the SONaFR's 
pf.conf file with network devices on his or her system 
(/etc/pf.conf): 


IntIf="nfe0" 
ExtIf="r1l0" 
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KarmaBSD 
| often meet with broken notebooks on which the CD-ROM 
or floppy drive does not work. It is a pity to discard such 
a notebook. If some sensitive data lies on its disk and the 
computer's hard drive does not boot, you will hardly find 
a BSD tool with use of which you can copy and save data 
from such a notebook's USB port to a USB hard drive (or 
stick). You may oppose that there is quite a number of 
BSD Live CD's today. But if the CD-ROM drive on such 
a notebook is broken, such an argument does not have 
any weight. 

KarmaBSD supports mounting of a number of disk 
formats: 

Type of medium: USB; CD-ROM/DVD; network 

Format: NTFS; ISO9660; EXT2FS; FAT MSDOS file 
systems; UDF; NFS file system (network) 


How to prepare a disk 

KarmaBSD, too, has the OpenBSD's base system tools 
such as newfs, disklabel, fdisk, etc., so if the hard drive 
(and the CD-ROM drive, too) on your notebook is broken, 
you may insert a new one into it and prepare your MS- 
DOS partition, for example: 


fdisk -e wd0 
then type: 
print 


(or help to learn more); then: 


edito (if 0 is the partition you want to edit); then: 04 (for 
DOS FAT16), or OB or OC (both for FAT32), or A6 (for 
OpenBSD), A5 (for FreeBSD), AQ (for NetBSD), 83 
(for Linux), 82 (for Linux swap). After you specify the 
operating system, type quit to save changes. 

Finally, the partition needs one more thing — a format. To 
make the FAT32 format on your hard drive, type: 


newfs msdos -F 32 /dev/rwd0i 


How to mount disks 
To mount any drive (NTFS drive, for example), type: 


disklabel wd0 
or 
disklabel sd0 


(USB disk) 


You will then see all available partitions marked with 
letters like j, k, I, m. 
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If you type: 
mount_ntfs /dev/wd0j /mnt 


your NTFS partition (;) will be mounted to the /mnt 
directory (as read only). 


Serial console 

This mini BSD also supports serial console (capturing 
output of a remote computer's screen to the screen of the 
local computer over a serial cable). Type: 


set tty com0 


at the boot prompt when KarmaBSD starts; then on your 
desktop computer (not the one on which KarmaBSD 
runs) type: 


tip tty00 


This way you can control any computer that has 
a broken keyboard or display (a very common failure 
with notebooks). Instead of the command tip ttyoo you 
must type in another OpenBSD box, you may use Putty, 
a free program also available for Windows. 


The most important thing - music 

KarmaBSD, too, can work as a one floppy MP3 player, 
as the purpose of this thing was to maximally utilize 
broken notebooks (computers). MP3 files can be played 
over network, too (NFS). It is all very simple. From within 
KarmaBSD you just type: 


The script (1 for /dev/caoa; Or 2 for /dev/caia) will mount 
your CD with MP3 files, it will create a playlist in /tmp, 
and with use of mpg123 you will automatically listen to 
hours of never-ending music or audiobooks by pressing 
one single key (1). 


How to put these floppy systems on a CD 

Today, when floppy drives gradually disappear from 
computers, both these one floppy systems may be quite 
easily put on a CD; to make the ISO image of any bootable 
diskette, just type dd (to copy the diskette's contents to 
a file — floppy.fs in our case); if you download the image 
of KarmaBSD or SONaFR, you do not have to do this, 
as you will have the same image you will create with the 
following command, but in case you first try the floppy, it 
may also be handy to do it this way): 
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dd if=/dev/fd0a of=floppy.fs 
Then run: 


mkisofs -b floppy.fs -c boot.cat -R -v -o / 
bootableCDfromFloppy.iso . 


(do not worry if "boot.cat" is not available) 

The dot (.) at the end of the above mkisofs 
command is for the current directory, so you must 
run this command in the (current) directory where the 
floppy.fs file resides. If floppy.fs is in the /£1p directory, 
for example (or uS@: da if=/dev/fd0a of=/flp/floppy.fs), 
cd to it (cd /£1p) and run the mkisofs command as you 
see above. 

The two above-mentioned one floppy systems can be 
thus used on a CD. To use multiboot, apply the -citorito- 
alt-boot option in mkisofs. 


How to add packages to KarmaBSD (or SONaFR) 
KarmaBSD is reviewed on the following forum: http:// 
www.daemonforums.org/showthread.php 7t=3092. 

The writer of the text in the above forum is interested to 
know how to add applications into KarmaBSD but does 
not know how, so | decided to add this information here. 

As some people think (as the author of the text in the 
forum) that it is a complication to install OpenBSD 4.1 
today — when the version 4.7 of OpenBSD is already 
available and the version 4.1 is rather quite old, | decided 
to publish a solution how packages can be instantly (and 
easily) transported from any OpenBSD (or FreeBSD) 
system to any OpenBSD (or FreeBSD, NetBSD, etc.) 
system. The solution is to compile packages statically. 
What does it mean? 





Figure 1. With Windows (running on another computer) and Putty 
you may connect to any computer (notebook) via a serial cable (serial 
console with KarmaBSD) and you may thus repair or control any 
computer 
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Ports and the base system 

Before you compile packages (from ports), you may 
decide to add the --enabie-static option to your makefiles. 
Without this, the result of compilation (binary application) 
of packages on OpenBSD 4.4, for example, depends 
always on this system's libraries. Every compilation 
is made against these global libraries (OpenBSD 4.7, 
FreeBSD 8.0, etc). Thus, if you compile packages in 
OpenBSD 4.7, which has different versions of libraries 
than OpenBSD 4.6, 4.5, etc., you cannot use these 
binaries in other versions of OpenBSD. 


Static versus dynamic 

The concept of the shared (dynamic) libraries (when 
many programs use certain libraries) started on the 
premise of saving space. But today, when a 40 GB hard 
drive is cheaper than ice cream in a very good European 
restaurant, the philosophy to save a few dozens of 
megabytes is, on the other hand, a very big barrier to 
portability of packages (if libraries are embedded in binary 
packages, you may use such binaries almost in any 
version of the system they were built for). 

Programs (and sources) contained in the base system 
(fdisk, ifconfig, chmod, etc.) are also dependent on the 
kernel they are distributed with. If you deal with sources 
of ifconfig (or ee editor in FreeBSD) in OpenBSD 4.7, for 
example, you must compile them against the OpenBSD 
4.7 kernel. 

This is a very important hint, as we must differentiate 
compilation of sources from 1) the base system (binaries 
are not easily transportable) and from 2) ports (add-on 
packages). The --enable-static option in your makefiles 
must only be used in source files that are not kernel- 
version dependent (/usr/ports). 

Base system tools like fdisk, mount, ifconfig, etc., are 
kernel(system)-version dependent (like OpenBSD 4.7) 
and although they, too, can be compiled statically, using 
them with a different kernel would bring a number of 
system failures. 

However, packages (in /usr/ports), i.e. binaries of 
ports compiled on OpenBSD 4.5, for example, are only 
library dependent (they require shared libraries used in 
OpenBSD 4.5), but not kernel dependent. They cannot 
be used on other versions of OpenBSD only because 
they were compiled against libraries that are missing 
in other versions of OpenBSD. To solve the issue 
(backward compatibility), you may install the so-called 
compatibility libraries. But if you work with a small floppy 
(or CD) system, it is a bit hard to install such compatibility 
libraries into the floppy (or minimal CD) environment. To 
solve this, compile packages (in /usr/ports) statically. 
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When you compile your sources this way (statically), 
the system's libraries get embedded into the resulting 
binaries. Packages (not the base system) statically 
compiled on OpenBSD 4.5, for example, will run on any 
OpenBSD system including KarmaBSD. The binary of 
mpg123 in KarmaBSD was compiled statically too. 

As most floppy systems are space limited, the best way 
is to make some additional room in memory and then 
copy such a statically compiled binary there from another 
floppy. 

KarmaBSD and SONaFR run in memory, too, so their 
floppies, after you boot with them, can be taken out. 

To prepare a memory file system in KarmaBSD, 
type (the number 4917 may vary in dependence of free 
RAM): 


mount_mfs -s 4917 -o async,nosuid,nodev,noatime swap /mnt 


The above two floppy systems can be downloaded here: 


« SONaFR - http://www. freebsd.nfo.sk/opbsd/ 
openbsdeng.htm 
« KarmaBSD- - _ http://www.freebsd.nfo.sk/opbsd/ 


karmabsdeng.htm 


They both were made with crunchgen. A crunched 
binary (result of crunchgen) is a program (kernel/system- 
version dependent) made up of many other programs 
(like fdisk, mount, chown, ifconfig, etc.) and libraries 
linked together into a single executable. The crunchgen 
utility is a tool that will make this binary. It is available 
on all BSD systems. And because it builds programs like 
fdisk, disklabel, mount, etc., from the heart of OpenBSD 
(or FreeBSD, etc.), you need sources of the base 
system. 

My first project was 1FCDBSD — One Floppy MP3 
and CD Player (KarmaBSD does not have a player for 
standard audio CD's). 1-FCDBSD is based on FreeBSD 
4.5. As | made a very handy multiboot CD with 1FCDBSD, 
SONaFR, and KarmaBSD, | am also happy that | can 
provide its download link. | thank www.rootbsd.net for 
allowing me to distribute my projects: 

ftp://2227.x.rootbsd.net/bsd-multiboot.iso 
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Remote Installation of the FreeBSD 


Operating System without a Remote Console 


This article documents the remote installation of the FreeBSD 
operating system when the console of the remote system is 
unavailable. The main idea behind this article is the result of 

a collaboration with Martin Matuska mm@FreeBSD.org with 
valuable input provided by Pawel Jakub Dawidek pjd@FreeBSD.org. 


What you will learn... 

+ you should have a network accessible operating system with SSH access 
* understand the FreeBSD installation process 

+ be familiar with the sysinstall(8) utility 

+ have the FreeBSD installation ISO image or CD handy 


world, but very few of them are officially supporting 
FreeBSD. They usually provide support for a Linux® 
distribution to be installed on the servers they offer. 
Insome cases, these companies will install your preferred 
Linux distribution if you request it. Using this option, we will 
attempt to install FreeBSD. In other cases, they may offer 
a rescue system which would be used in an emergency. It's 
possible to use this for our purposes as well. 
This article covers the basic installation and configuration 
steps required to bootstrap a remote installation of 
FreeBSD with RAID-1 and ZFS capabilities. 


T are many server hosting providers in the 


Introduction 

This section will summarize the purpose of this article and 
better explain what is covered herein. The instructions 
included in this article will benefit those using services 
provided by colocation facilities not supporting FreeBSD. 


« As we have mentioned in the Background section, 
many of the reputable server hosting companies 
provide some kind of rescue system, which is booted 
from their LAN and accessible over SSH. They 
usually provide this support in order to help their 
customers fix broken operating systems. As_ this 
article will explain, it is possible to install FreeBSD 
with the help of these rescue systems. 





What you should know... 
you will know how to remotely install (using SSH) a FreeBSD system from 
within other operating system, such as Linux. It covers an advanced 
installation procedure with which one can replace Linux installation 
with FreeBSD OS. 


e« The next section of this article will describe how to 
configure, and build minimalistic FreeBSD on the 
local machine. That version will eventually be running 
on the remote machine from a ramdisk, which will 
allow us to install a complete FreeBSD operating 
system from an FTP mirror using the sysinstall utility. 

¢ The rest of this article will describe the installation 
procedure itself, as well as the configuration of the 
ZFS file system. 


Requirements 
To continue successfully, you must: 


* Have a network accessible operating system with 
SSH access 

e Understand the FreeBSD installation process 

¢ Be familiar with the sysinsta11(8) utility 

« Have the FreeBSD installation ISO image or CD 
handy 


Preparation - mfsBSD 

Before FreeBSD may be installed on the target system, 
it is necessary to build the minimal FreeBSD operating 
system image which will boot from the hard drive. This 
way the new system can be accessed from the network, 
and the rest of the installation can be done without remote 
access to the system console. 
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The mfsBSD tool-set can be used to build a tiny FreeBSD 
image. As the name of mfsBSD suggests (mfs means 
memory file system), the resulting image runs entirely from 
a ramdisk. Thanks to this feature, the manipulation of hard 
drives will not be limited, therefore it will be possible to 
install a complete FreeBSD operating system. The home 
page of mfsBSD, at hitp:/people.freebsd.org/~mm/mfsbsd/ 
, includes pointers to the latest release of the toolset. 

Please note that the internals of mfsBSD and how 
it all fits together is beyond the scope of this article. 
The interested reader should consult the original 
documentation of mfsBSD for more details. 

Download and extract the latest mfsBSD release and 
change your working directory to the directory where the 
mfsBSD scripts will reside: 


# fetch http://people.freebsd.org/~mm/mfsbsd/mfsbsd- 
latest.tar.gz 

# tar xvzf mfsbsd-1.0-beta3.tar.gz 

# cd mfsbsd-1.0-beta3/ 


Configuration of mfsBSD 

Before booting mfsBSD, a few important configuration 
options have to be set. The most important that we have 
to get right is, naturally, the network setup. The most 
suitable method to configure networking options depends 
on whether we know beforehand the type of the network 
interface we will use, and the network interface driver to 
be loaded for our hardware. We will see how mfsBSD can 
be configured in either case. 

Another important thing to set is the root password. This 
can be done by editing the cont/rootpw.cont file. Please keep 
in mind that the file will contain your password in the plain 
text, thus we do not recommend to use real password here. 
Nevertheless, this is just a temporary one-time password 
which can be later changed in a live system. 


The conf/interfaces.conf method 

When the installed network interface card is unknown, 
we can use the auto-detection features of mfsBSD. The 
startup scripts of mfsBSD can detect the correct driver to 
use, based on the MAC address of the interface, if we set 
the following options in conf/interfaces.conf! 


initconf_interfaces="extl" 
initconf_mac_ext1="00:00:00:00:00:00" 
initconf_ip_ext1="192.168.0.2" 
initconf_netmask_ext1="255.255.255.0" 


Do not forget to add the defaultrouter information to the 


conf/re.cont file: 
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defaultrouter="192.168.0.1" 


The conf/rc.conf method 
When the network interface driver is known, it is more 
convenient to use the conf/rc.cont file for networking 
options. The syntax of this file is the same as the one used 
in the standard rc.cone 5) file of FreeBSD. 

For example, if you know that a re (4) network interface 
is going to be available, you can set the following options 


IN conf/re. conf: 


defaultrouter="192.168.0.1" 
ifconfig_re0="inet 192.168.0.2 netmask 255.255.255.0" 


Building an mfsBSD image 
The process of building an mfsBSD image is pretty 
straightforward. 

The first step is to mount the FreeBSD installation CD, 
or the installation ISO image to /carom. For the sake of 
example, in this article we will assume that you have 
downloaded the FreeBSD 8.0-RELEASE ISO. Mounting 
this ISO image to the /carom directory is easy with the 
mdconfig (8) Utility: 


# mdconfig -a -t vnode -u 10 -f 8.0-RELEASE-amd64-discl.iso 
# mount_cd9660 /dev/md10 /cdrom 


Next, build the bootable mfsBSD image: 
# make BASE=/cdrom/8 .0-RELEASE 


Note: The above make command has to be run from the top 
level of the mfsBSD directory tree, i.e. ~/mfsbsd-1.0-beta3/. 


Booting mfsBSD 

Now that the mfsBSD image is ready, it must be uploaded 
to the remote system running a live rescue system or pre- 
installed Linux® distribution. The most suitable tool for 
this task is scp: 


# scp disk.img root@192.168.0.2:. 

To boot mfsBSD image properly, it must be placed on the 
first (bootable) device of the given machine. This may be 
accomplished using this example providing that sda is 
the first bootable disk device: 

# dd if=/root/disk.img of=/dev/sda bs=1m 

If all went well, the image should now be in the MBR of the 


first device and the machine can be rebooted. Watch for 
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the machine to boot up properly with the ping s) tool. Once 
it has came back on-line, it should be possible to access it 
over ssn(1) aS user root with the configured password. 


Installation of The FreeBSD Operating System 
The mfsBSD has been successfully booted and it should 
be possible to log in through ssh(1). This section will 
describe how to create and label slices, set up gmirror 
for RAID-1, and how to use sysinstall to install a minimal 
distribution of the FreeBSD operating system. 


Preparation of Hard Drives 
The first task is to allocate disk space for FreeBSD, i.e.: 
to create slices and partitions. Obviously, the currently 
running system is fully loaded in system memory and 
therefore there will be no problems with manipulating hard 
drives. To complete this task, it is possible to use either 
sysinstall or ¢aisk (8) in conjunction to bsatabel (8). 

At the start, mark all system disks as empty. Repeat the 
following command for each hard drive: 


# dd if=/dev/zero of=/dev/ad0 count=2 


Next, create slices and label them with your preferred 
tool. While it is considered easier to use sysinstall, 
a powerful and also probably less buggy method will 
be to use standard text-based UNIX® tools, such as 
fdisk(8) ANd psalabe1(s), Which will also be covered in 
this section. The former option is well documented in the 
Installing FreeBSD chapter of the FreeBSD Handbook. 
As it was mentioned in the introduction, this article will 
present how to set up a system with RAID-1 and ZFS 
capabilities. Our set up will consist of a small gmirror(s) 
mirrored / (root), /usr and /var file systems, and the rest 
of the disk space will be allocated for a zpoo1(s) mirrored 
ZFS file system. Please note, that the ZFS file system 
will be configured after the FreeBSD operating system is 
successfully installed and booted. 

The following example will describe how to create slices 
and labels, initialize omirror(s) On each partition and how 
to create a UFS2 file system in each mirrored partition: 


fdisk -BI /dev/adco @ 

fdisk -BI /dev/adl 

bsdlabel -wB /dev/ad0s1 @ 

bsdlabel -wB /dev/adisl 

bsdlabel -e /dev/ad0s1 

bsdlabel /dev/ad0sl > /tmp/bsdlabel.txt && bsdlabel -R 
/dev/adis1 /tmp/bsdlabel.txt e 

gmirror label root /dev/ad[01]sla 6 


Se HE SESE HEHE 


Se te 


gmirror label var /dev/ad[01]sld 
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gmirror label usr /dev/ad[01]sle 
gmirror label -F swap /dev/ad[01]slb @ 
newfs /dev/mirror/root 


newfs /dev/mirror/var 


SE HE HEHEHE 


newfs /dev/mirror/usr 


@create (http://www.freebsd.org/doc/en_US.ISO8859- 
V/articles/remote-install/installation.html#FDISK) a slice 
covering the entire disk and initialize the boot code 
contained in sector 0 of the given disk. Repeat this 
command for all hard drives in the system. 

@ write (http://www. freebsd.org/doc/en_US.ISO8859- 
1/articles/remote-install/installation.html#BSDLABEL- 
WRITING) a standard label for each disk including the 
bootstrap code. 

Now _ (htto:/;www.freebsd.org/doc/en_US.ISO8859- 
1/articles/remote-install/installation.html#BSDLABEL- 
EDITING), manually edit the label of the given disk. Refer 
to the bsdlabei(s) (Attp:/www.freebsd.org/cgi/man.cgi?qu 
ery=bsdlabel&sektion=8) manual page in order to find out 
how to create partitions. Create partitions a for / (root) file 
system, b for swap, d for /var, e for /usr and finally f which 
will later be used for ZFS. 

Import (http:/Awww.freebsd.org/doc/en_US.I|SO8859- 
1/articles/remote-install/installation.html#BSDLABEL- 
RESTORE) the recently created label for the second hard 
drive, so both hard drives will be labeled in the same way. 
@ initialize (http:/www.freebsd.org/doc/en_US.I|SO8859- 
1/articles/remote-install/installation.html#GMIRRORT) 
gmirror (8) (http:/Avww.freebsd.org/cgi/man.cgi?query=gmir 
ror&sektion=8) on each partition. 

@ Note (http://www. freebsd.org/doc/en_US.I|SO8859-1/ 
articles/remote-install/installation.htmil#GMIRROR2) the - 
F option used for swap partition. This instructs gmirror(s) 
(http://www. freebsd.org/cgi/man.cgi?query=gmirror&sekti 
on=8) to assume that the device is in the consistent state 
after the power/system failure. 

@ Create (http://www. freebsd.org/doc/en_US.ISO8859- 
1/articles/remote-install/installation.htmlENEWFS) 
a UFS2 file system on each mirrored partition. 


System Installation 

This is the most important part. This section will describe 
how to actually install the minimal distribution of FreeBSD 
on the hard drives that we have prepared in the previous 
section. To accomplish this goal, all file systems need 
to be mounted so sysinstall may write the contents of 
FreeBSD to the hard drives: 


# mount /dev/mirror/root /mnt 


# mkdir /mnt/var /mnt/usr 
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# mount /dev/mirror/var /mnt/var 


# mount /dev/mirror/usr /mnt/usr 


When you are done, start sysinsta1i(s). Select the 
Custom installation from the main menu. Select Options 
and press Enter. With the help of arrow keys, move the 
cursor on the Install Root item, press Space and change 
it to /mnt. Press Enter to submit your changes and exit 
the Options menu by pressing q. 

Warning: Note that this step is very important and if 
skipped, sysinstall will be unable to install FreeBSD. 

Go to the Distributions menu, move the cursor with the 
arrow keys on the Minimal option, and check it by pressing 
Space. This article uses the Minimal distribution in order 
to save network traffic, because the system itself will be 
installed over ftp. Exit this menu by choosing Exit option. 

Note: The Partition and Label menus will be skipped, as 
these are useless now. 

Inthe Media menu, select FTP. Select the nearest mirror and 
let sysinstall assume that the network is already configured. 
You will be returned back to the Custom menu. Finally, 
perform the system installation by selecting the last option, 
Commit. Exit sysinstall when it finishes the installation. 


Post Installation Steps 

The FreeBSD operating system should be installed now; 
however, the process is not finished yet. It is necessary 
to perform some post installation steps in order to allow 
FreeBSD to boot in the future and to be able to log in 
to the system. You must now chroot s) into the freshly 
installed system in order to finish the installation. Use the 
following command: 


# chroot /mnt 
To complete our goal, perform these steps: 


* Copy the GENERIC kernel to the /boot/kerne1 directory: 
# cp -Rp /boot/GENERIC/* /boot/kernel 
* Create the /etc/rc.cont, and 
fstab files. Do not forget to properly set the network 
information and to enable sshd in the /etc/rc.con¢ file. 
The contents of the /etc/fstab file will be similar to the 
following: 


/etc/resolv.conf J/etc/ 


# Device Mountpoint FStype Options Dump Pass# 
/dev/mirror/swap none swap sw 0 0 
/dev/mirror/root / ufs rw ai a 
/dev/mirror/usr /usr ufs rw 2 2 
/dev/mirror/var /var ufs rw 2 2 
/dev/cd0d /cdrom cd9660 ro,noauto 0 0 
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Create the /boot/loader.conf file, with the following 
contents: 

geom_mirror_load="YES" 

zfs_load="YES" 
Perform the following command, which will make ZFS 
available on the next boot: 
# echo 'zfs_enable="YES"' >> /etc/rc.conf 
Add additional users to the system using the adduser(8) 
tool. Do not forget to add a user to the wheel group so 
you may obtain root access after the reboot. 
Double-check all your settings. 


The system should now be ready for the next boot. Use 
the reboot (8) Command to reboot your system. 


ZFS 

If your system survived the reboot, it should now be 
possible to log in. Welcome to the fresh FreeBSD 
installation, performed remotely without the use of 
a remote console! 

The only remaining step is to configure zpo01(8) 
and create some zs s) file systems. Creating and 
administering ZFS is very straightforward. First, create 
a mirrored pool: 


# zpool create tank mirror /dev/ad[01]slf 
Next, create some file systems: 


# zfs create tank/ports 

# zfs create tank/src 

# zfs set compression=gzip tank/ports 

# zfs set compression=on tank/src 

# zfs set mountpoint=/usr/ports tank/ports 
# zfs set mountpoint=/usr/sre tank/srce 


That's all. If you are interested in more details about 
ZFS on FreeBSD, please refer to the ZFS (http:/ 
wiki.freebsd.org/ZFS) section of the FreeBSD Wiki. 
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HOW TO’S 


OpenBSD 


as a Mail Server 


In a previous document, we built redundant firewalls using the 
CARP and PFSYNC protocols; these were the first building blocks 
of a hypothetical, OpenBSD-based, small private networkthat we 
are going to build step by step across several documents. 


What you will learn... What you should know... 
+ Installing a full-featured mail server + Agood knowledge of OpenBSD administration 
+ Basic mail server security + Basic MySQL database administration 





network, it's time to think about the services we 

want to provide. Offering a reliable and secure 
email service is probably one of the top priorities of most 
system administrators; therefore, in the next chapters, 
we will build a full-featured mail server, based on open- 
source software and focusing on security. The following is 
the list of the pieces of software we will use: 


N ow that we have raised the defensive walls of our 


OpenBSD 

http://www.openbsd.org/ — the secure by default operating 
system, with only two remote holes in the default install, in 
a heck of a long time!; 





Postfix 

http://www.postfix.org/ — an MTA that started life at IBM 
research as an alternative to the widely-used Sendmail 
(http:/www.sendmail.org/) program and which attempts to 
be fast, easy to administer, and secure; 


MySQL 
http://www.mysql.com/ — the world's most popular open 
source database; 


Courier-IMAP 


http://www.courier-mta.org/imap/ — a fast, scalable, enterprise 
IMAP server that supports MySQL and maildirs; Figure 1. Network layout 
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Cyrus SASL 

http://asg.web.cmu.edu/sasl/ -— the Cyrus (http:// 
cyrusimap.web.cmu.edu/) implementation of the SASL 
(http://asg.web.cmu.edu/sasl/) protocol; 


Amavisd-new 

http://www. ijs.si/software/amavisd/ — a high-performance 
interface between mailer (MTA) and content checkers 
(antivirus and antispam), written in Perl and optimized for 
Postfix; 


SpamAssassin 

http://spamassassin.apache.org/ — a Perl-based mail 
filter to identify Spam, using a variety of mechanisms 
including header and text analysis, Bayesian filtering, 
DNS blocklists, and collaborative filtering databases; 


ClamAV 
http://www.clamav.net/ — a fast and easy-to-use open- 
source virus scanner. 

A good knowledge of OpenBSD is assumed, since we 
won't delve into system management topics such as base 
configuration (http://www. openbsd.org/cgi-bin/man.cgi 
?query=afterboot&sektion=8) or packages/ports (http:// 
www.openbsd.org/faq/faq15.html) installation. 


Preliminary installation steps 

Before delving into the installation and configuration of all 
the mail-handling software, we will take a brief look at the 
operating system that will host it. 

As usual, my choice goes to OpenBSD for its proven 
security, reliability and ease of use. Needless to say, all 
these features are essential for a system that will have to 
handle a large volume of email traffic while still making life 
hard for spammers and malicious users. 

We won't dwell upon the installation procedure here, 
which is documented in full detail on the OpenBSD web 
site (http:/www.openbsd.org/faq/faq4. html). Just a couple 
of notes: 


¢ while partitioning the hard drive, bear in mind that 
we will configure Postfix to use virtual domains 
(http://www. postfix.org/VIRTUAL_README. html!) 
and, consequently, it will store all users' mail 
folders in a single directory (/var/vmaii). Therefore, 
it is recommended to assign a (large) dedicated 
slice to this filesystem, in order to prevent mails 
from filling up any critical filesystem, should quotas 
fail. Furthermore, if you choose to install MySQL on 
the mail server itself, it is usually recommended to 
assign one of the first slices to /var/mysqi, in order 


www.bsdmag.org 


to allow for faster disk access by the database 
engine; 

« the only file sets we will need to install are those 
marked as required on the documentation (http:// 
www.openbsd.org/faq/faq4.html#FilesNeeded), i.e. 
bsd (the kernel), basexx.tgz (the base system), and 
etcxx.tgz (the configuration files in /etc) plUS compxx.tgz 
(the C compiler), since we will also have to install 
some ports = (http://www.openbsd.org/ports. htm!) 
not available as precompiled packages for licensing 
reasons. 


Note: since leaving a compiler on a publicly accessible 
server is a definite security risk, it is recommended that 
you remove the compiler when the installation is over or 
that you compile on another machine. 

After the first reboot, we can disable some default 
network services managed by _inetd(8)  (http:// 
www.openbsd.org/cgi-bin/man.cgi?query=inetd&sektion= 
8): see Listing 1. 

by commenting them out in /etc/inetd.cont (http:// 
www.openbsdorg.com/cgi-bin/man.cgi?query=inetd&sekt 
ion=8) and reloading ineta 8) (http://www.openbsdorg.com/ 
cgi-bin/man.cgi?query=inetd&sektion=8): 


# pkill -HUP inetd 





Listing 1. Default services on OpenBSD 
§ grep -v *# /etc/inetd.conf 
ident stream tcp nowait _identd /usr/ 
libexec/identd identd -el 
ident stream tcp6 nowait _identd /usr/ 
libexec/identd identd -el 
127.0.0.1:comsat dgram udp wait root /usr/ 
libexec/comsat comsat 
[=sieicomsat dgram udp6 wait root /usr/ 
libexec/comsat comsat 
daytime stream tcp nowait root 
internal 
daytime stream tcp6 nowait root 
internal 
time stream tcp nowait root 
internal 
time stream tcp6 nowait root 
internal 
$ 
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Listing 2. Postfix configuration 


/etc/postfix/main.cf 

# Directory containing all the post* commands 

command_directory = /usr/local/sbin 

# Directory containing all the Postfix daemon programs 

daemon directory = /usr/local/libexec/postfix 

# Full pathnames of various Postfix commands 

sendmail path = /usr/local/sbin/sendmail 

newaliases path = /usr/local/sbin/newaliases 

mailg path = /usr/local/sbin/mailg 

# Directories containing documentation 

html_directory = /usr/local/share/doc/postfix/html 

manpage directory = /usr/local/man 

readme directory = /usr/local/share/doc/postfix/readme 

# The owner of the Postfix queue and of most Postfix 
daemon processes 

mail_owner = _postfix 

# The group for mail submission and queue management 
commands 

setgid_ group = _postdrop 

# The myhostname parameter specifies the internet 
hostname of this mail system. It 

# is used as default for many other configuration 
parameters (default = system's 

# FODN) 

myhostname = mail.kernel-panic.it 

# The internet domain name of this mail system. Used 
as default for many other 

# configuration parameters (default = Smyhostname minus 
the first component) 

mydomain = kernel-panic.it 

# The domain name that locally-posted mail appears to 
come from, and that locally 

# posted mail is delivered to. As you can see, 
a parameter value may refer to other 

# parameters 

myorigin = $myhostname 

# Network interface addresses that this mail system 
receives mail on 

inet interfaces = all 

# Network interface addresses that this mail system 
receives mail on by way of a 

# proxy or NAT unit 

proxy interfaces = router.kernel-panic.it 

# List of domains that this machine considers itself 
the final destination for. 

# Virtual domains must not be specified here 


mydestination = $myhostname, localhost.$mydomain, localhost 


# List of "trusted" SMTP clients allowed to relay mail 
through Postfix. 

mynetworks = 127.0.0.0/8, 172.16.0.0/24, 172.16.240.0/24 

# What destination (sub)domains this system will relay 
mail to 

relay domains = $mydestination 

# The default host to send mail to when no entry is 
matched in the optional 

# transport (5) table. Square brackets turn off MX lookups 

relayhost = [smtp.isp.com] 

# List of alias databases used by the local delivery 
agent 

alias maps = hash:/etc/postfix/aliases 

# Alias database(s) built with "newaliases" or 
"sendmail -bi". This is a separate 

# configuration parameter, because alias maps may 
specify tables that are not 

# necessarily all under control by Postfix 

alias database = hash:/etc/postfix/aliases 

# SMTP greeting banner 

smtpd_banner = $myhostname ESMTP $mail_ name 

# Postfix is final destination for the specified list of 
"virtual" domains 

virtual_mailbox_domains = kernel-panic.it 

# Virtual mailboxes base directory 

virtual mailbox base = /var/vmail 

# Optional lookup tables with all valid addresses in 
the domains that match 

# $virtual_mailbox domains. 

virtual mailbox maps = hash:/etc/postfix/vmailbox 

# The minimum user ID value accepted by the virtual (8) 
delivery agent 

virtual_minimum_uid = 2000 

# User ID that the virtual(8) delivery agent uses 
while writing to the recipient's 

# mailbox 

virtual uid maps = static:2000 

# Group ID that the virtual(8) delivery agent uses 
while writing to the recipient's 

# mailbox 


virtual_gid_maps = static:2000 


# Optional lookup tables that alias specific mail 
addresses or domains to other 
# local or remote address 


virtual alias maps = hash:/etc/postfix/virtual 
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Anyway, OpenBSD is considered secure also with those 
services turned on and the mail server should be placed 
behind a firewall; nevertheless, | prefer staying on the 
safe side and disable them all (including comsat:s) (http: 
//www.openbsdorg.com/cgi-bin/man.cgi?query=comsa 
t&sektion=8), since we won't have any interactive user 
receiving mail on the system). 

To modify the server network configuration, please 
refer to the related chapter (http:/www.kernel-panic. it/ 
openbsd/carp/carp3.html) in the previous document about 
redundant firewalls (http:/;vwww.kernel-panic.it/openbsd/ 
carp/) or to the networking (htto:/Avww.openbsd.org/faq/ 
faq6.html) FAQ. 


Postfix 

Postfix (http:/www.postfix.org/)) is a MTA (Mail 
Transport Agent) developed by Wietse Venema (http: 
//www.porcupine.org/wietse/) as an alternative to the 
widely-used Sendmail (http://www.sendmail.org/) 
program. \t attempts to be fast, easy to administer, and 
secure, while at the same time being sendmail compatible 
enough to not upset existing users. Thus, the outside has 
a sendmail-ish flavor, but the inside is completely different. 
Postfix also comes with excellent documentation (http:// 
www. postfix.org/documentation.html) and a lot of howtos 
(http:/www. posttix.org/docs.htm!). 

Our mail server requirements will be quite simple: it will 
be final destination solely for its canonical domains (http: 
//www.postfix.org/docs.html) and it will only relay mail 
from systems on the internal network (though we will also 
consider relaying from untrusted networks by means of 
SMTP. authentication). Canonical domains include the 
hostname (in our case, mail.kernel-panic.it) and the IP 
address (172.16.240.150) of the machine that Postfix 
runs on, and the parent domain of the hostname (xerne1- 
panic.it). 

Canonical domains are usually implemented with the 
Postfix local domain address class (http://www. postfix.org/ 
ADDRESS_CLASS_README.html#local_domain_ 
class), which, unfortunately, has one major drawback 
for me: it requires that each e-mail account have 
a corresponding Unix account. On the contrary, | prefer: 


1. keeping Unix and e-mail accounts apart and 
2. having all mailboxes well-ordered inside a single 
directory. 


Therefore, we will use Postfix Virtual Domain Hosting (http: 
//www.postfix.org/VIRTUAL_README.html), which _ is 
normally used for hosting multiple internet domains on the 
same server, but will also allow us to achieve our two goals. 
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Configuration 

In this paragraph, we will configure Postfix to work 
standalone, with no back-end database. Then, in the 
next chapter, when everything will be working fine, we will 
hook up Postfix to a MySQL database; this will allow us to 
centrally store configuration information that both Postfix 
and Courier-IMAP will need to access. 

There are a few packages we need to install: 

° mysql-client-x.x.x.tgz 
°  pcre-x.x.tgz 


°  postfix-x.x.x-mysql.tgz 

Note: if you're planning to use SMTP authentication, 
you will need to compile Postfix from the ports, because 
there's no pre-compiled package available with both 
MySQL and SASL support: 





Listing 3. Replacing Sendmail with Postfix 


# /usr/local/sbin/postfix-enable 
old /etc/mailer.conf saved as /etc/mailer.conf.pre- 
postfix 


postfix /etc/mailer.conf enabled 


NOTE: do not forget to add sendmail flags="-bd" to 
fetc/rc.conf.local to startup postfix correctly. 
NOTE: do not forget to add "-a /var/spool/postfix/dev/ 
bog” re 
syslogd flags in /etc/rc.conf.local and restart 
syslogd. 
NOTE: do not forget to remove the "sendmail 


clientmqueue runner" 
from root's crontab. 


# 


Listing 4. Postfix startup commands 


/etc/re.conf.local 

# Specify a location where syslogd(8) should place an 
additional log socket 

# for Postfix 

syslogd flags="-a /var/spool/postfix/dev/log" 


# Make Postfix start in background and process queued 
messages every 30 min 


sendmail flags="-bd" 
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# cd /usr/ports/mail/postfix/snapshot 
# env FLAVOR="mysql sasl2" make install 


The installation will create the /etc/posttix directory, 
containing all the configuration files. Postfix has 
several hundred configuration parameters that are 
controlled via the /etc/postfix/main.ct file, but don't 


worry: for the vast majority of these parameters, the 
default value is the best option (see postcons(s) (http: 
//www.posttfix.org/postconf.5.html) for a detailed list 
of all the available configuration parameters, their 
description and their default value) and we will only 
have to override a very small subset of them: see 
Listing 2. 





Listing 5. Testing the basic functionality 


# chgrp _postdrop /usr/local/sbin/postqueue /usr/ 
local/sbin/postdrop 

# chmod 2755 /usr/local/sbin/postqueue /usr/local/ 
sbin/postdrop 

# pkill syslogd 

# syslogd -a /var/empty/dev/log -a /var/spool/posttfix/ 
dev/log 

# pkill sendmail 

# /usr/local/sbin/sendmail -bd 

postfix/postfix-script: starting the Postfix mail system 

and test our hard work! 

# telnet mail.kernel-panic.it 25 

Trying Li2.16.240. 150... 

Connected to mail.kernel-panic.it. 

Escape character is '*]'. 

220 mail.kernel-panic.it ESMTP Postfix 

HELO somedomain.org 

250 mail.kernel-panic.it 

mail from: someone@somedomain.org 

250 Ok 

rept to: d.mazzocchio@kernel-panic.it 

250 Ok 

data 

354 End data with <CR><LF>.<CR><LF> 

From: someone@somedomain.org 

To: d.mazzocchio@kernel-panic.it 


Subject: Test mail 
It works! 


250 Ok: queued as 548D7286 

quit 

221 Bye 

Connection closed by foreign host. 

# tail /var/log/maillog 

Dec 16 10 15:26:35 mail postfix/smtpd[29212]: 
connect from wsl.lan.kernel- 
panic .it (hie. 86.0. 15] 

Dec 16 15:26:53 mail postfix/smtpd[29212]: 57076222: 





client=wsl.lan.kernel- 
penie. ce Lies oO] 

Dec 16 15:27:02 mail postfix/cleanup[13428]: 57076222: 
message-id=<20070210142653.5707622 
2@mail.kernel-panic.it> 

Dec 16 15:27:02 mail postfix/gmgr [26776]: 57076222: 
from=<someone@somedomain.org>, 
size=392, nrcpt=1 (queue active) 

Dec 16 15:27:02 mail postfix/virtual [14381]: 57076222: 
to=<d.mazzocchio@kernel-panic.it>, 
relay=virtual, delay=15, 
delays=15/0.28/0/0.03, dsn=2.0.0, 
status=sent (delivered to maildir) 

Dec 16 15:27:02 mail postfix/gmgr[26776]: 57076222: 
removed 

Dec 16 15:27:06 mail postfix/smtpd[29212]: 
disconnect from wsl.lan.kernel- 
Canicsiel le. Poovey 

# cat /var/vmail/kernel-panic.it/d.mazzocchio/new/1118 
146014.V3I9448M811660.mail 

.kernel-panic.it 

Return-Path: <someone@somedomain.org> 

X-Original-To: d.mazzocchio@kernel-panic.it 

Delivered-To: d.mazzocchio@kernel-panic.it 

Received: from somedomain.org (wsl.lan.kernel-panic.it 
V2 Gr Oe eSy))) 

by mail.kernel-panic.it (Postfix) with SMTP id 
57076222 

for <d.mazzocchio@kernel-panic.it> Sat, 16 Dec 
2007 15:26:47 +0100 (CET) 

From: someone@somedomain.org 

To: d.mazzocchio@kernel-panic.it 

Subject: Test mail 

Message-Id: <20070210142653.57076222@mail.kernel- 
Pane siete 

Date: Sat, 16 Dec 2007 15:26:47 +0100 (CET) 


It works! 
# 
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Let's take a closer look at some of the above 
configuration parameters. 

One of the goals we had was to avoid having 
a separate Unix account for each e-mail account. We 
have achieved this by configuring Postfix to write to 
the mailboxes using uid 2000 and gid 2000 (see the 
virtual _uid maps ANd virtual gid maps parameters above). 
Now we only have to create a user with this pair of uid 


and gid: 


# useradd -d /var/vmail -g =uid -u 2000 -s /sbin/nologin \ 


> -c "Virtual Mailboxes Owner" -m vmail 


Our second goal was having all mailboxes grouped 
together in a single directory; this is achieved by setting 
the value of the virtual mailbox base parameter to the 
path of that directory (in our configuration, /var/vmaii). 
In matter of fact, this parameter is a prefix that the 
virtual(s) (http://www.postfix.org/virtual.8.html) agent 
prepends to all pathname results from virtual _ mailbox _ 
maps table lookups. 

In our configuration, the virtual mailbox maps parameter 
refers to the file, containing 
the list of all valid addresses in the virtual domains 
(virtual_mailbox domains parameter) and the path to 
the corresponding mailboxes or maildirs (a mailbox is 
a single file containing all the emails; a maildir (http: 
/www.qmail.org/man/man5/maildir.html), instead, is 
a directory, with a defined structure, containing all the 
emails in separate files): 


/etc/postf£ix/vmailbox 


/etc/postfix/vmailbox 
info@kernel-panic.it kernel-panic.it/info/ 
d.mazzocchio@kernel-panic.it kernel-panic.it/ 


d.mazzocchio/ 


Please pay attention to the trailing slashes: they tell 
Postfix that the pathname refers to a maildir instead of 
a mailbox file, and maildirs are our only option, since 
Courier-IMAP doesn't support mailbox files. 

The virtual _alias maps parameter allows to alias specific 
mail addresses or domains to other local or remote 
address. Its value is the pathname to a file (in our case 
/etc/postfix/virtual) containing the alias mappings: 


/etc/postfix/virtual 

root@kernel-panic.it root@localhost.kernel- 
panic.it 

postmaster@kernel-panic.it postmaster@localhost.kerne 


l-panic.it 


www.bsdmag.org 


abuse@kernel-panic.it postmaster@localhost.kerne 


l=panic.it 


Finally, the /etc/posttfix/aliases file contains the addresses 
to which Postfix will redirect mail for local recipients (see 
aliases(5) http:/Awww.postfix.org/aliases.5.html). Since 
many accounts point to root's email address, you should 
check root email frequently or forward it all to another 
account. E.g.: 


/etc/postfix/aliases 
root: d.mazzocchio@kernel-panic.it 
[AILER-DAEMON: postmaster 


postmaster: root 





Listing 6. MySQL installation and configuration 


# /usr/local/bin/mysql_install_db 

# mysqld_safe & 

# /usr/local/bin/mysql_ secure installation 

Enter current password for root (enter for none): 

<Enter> 

OK, successfully used password, moving on... 

Set root password? [Y/n] Y 

New password: root 

Re-enter new password: root 

Password updated successfully! 

Remove anonymous users? [Y/n] Y 
«= success ! 

Disallow root login remotely? [Y/n] Y 


= success ! 


Remove test database and access to it? [Y/n] Y 
- Dropping test database... 
- success! 
- Removing privileges on test database... 
. success! 
Reload privilege tables now? [Y/n] Y 


; suecess ! 
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bin: root 


Now we only have to reload Postfix lookup tables: 


# /usr/local/sbin/postmap /etc/postfix/vmailbox 
# /usr/local/sbin/postmap /etc/postfix/virtual 


# /usr/local/sbin/newaliases 


replace Sendmail: see Listing 3. 
and follow the above advice, by commenting out the 
sendmail clientmqueue runner in root's crontab: 


# sendmail clientmqueue runner 


#*/30 * * * * /usr/sbin/sendmail -L sm-msp-queue -Ac -q 


and adding a couple of variables in the /etc/ 
rc.conf.local(8) (http://www.openbsd.org/cgi-bin/man.cgi 
?query=rc.conf.local&sektion=8) file see Listing 4. 

Now we can change a few permissions and restart the 
processes (or simply reboot): see Listing 5. 


MySQL 

If Postfix is working fine, we can proceed to the next 
step and install MySQL. MySQL is the world's most 
popular open source database, combining performance, 
reliability and ease of use. It will ensure faster data 
access times and allow us to centralize configuration 
information that both Postfix and Courier-IMAP will need 
to access. 

There are a few packages we need to install: 





Listing 7. Creating the database 


# mysql -u root -p 

password: root 

mysql> CREATE DATABASE mail; 

Query OK, 1 row affected (0.01 sec) 


mysql> use mail 
Database changed 
mysql> CREATE TABLE domains ( 


=> id INT NOT NULL PRIMARY KEY AUTO_ 
INCREMENT, 
=> domain VARCHAR (255) NOT NULL UNIQUE) ; 


Query OK, 0 rows affected (0.02 sec) 


mysql> CREATE TABLE users ( 














==> id INT NOT NULL PRIMARY KEY AUTO _ 
INCREMENT, 

=> login VARCHAR (255) NOT NULL UNIQUE, 

-> name VARCHAR (255) NOT NULL, 

-> password CHAR(13) NOT NULL, 

=> uid SMALLINT NOT NULL DEFAULT 2000, 

=> gid SMALLINT NOT NULL DEFAULT 2000, 

==> home VARCHAR (255) NOT NULL DEFAULT 
'/var/vmail', 

=> maildir VARCHAR(255) NOT NULL, 

=> quota VARCHAR(10) NOT NULL DEFAULT 
'10000000S') ; 


Query OK, 0 rows affected (0.01 sec) 


mysql> CREATE TABLE alias _ maps ( 
=> a INT NOT NULL PRIMARY KEY AUTO_ 
INCREMENT, 





=> account VARCHAR(255) NOT NULL UNIQUE, 
-> alias VARCHAR (255) NOT NULL) ; 
Query OK, 0 rows affected (0.00 sec) 


mysql> GRANT SELECT ON mail.* to 'vmail'@'localhost' 
IDENTIFIED BY 'vmail'; 
Query OK, 0 rows affected (0.01 sec) 


mysql> INSERT INTO domains (domain) VALUES ('kernel- 
Pancras!) 


Query OK, 1 row affected (0.01 sec) 


mysql> INSERT INTO users (login, name, password, 
maildir) 
-> VALUES ('d.mazzocchio@kernel-panic.it', 
"Daniele Mazzocchio', 
-> ENCRYPT('danix'), 'kernel-panic.it/ 
d.mazzocchio/'); 


Query OK, 1 row affected (0.01 sec) 


mysql> INSERT INTO alias maps (account, alias) 
-> VALUES ('postmaster@kernel-panic.it', 
=> "postmaster@localhost.kernel- 
Pancha 


Query OK, 1 row affected (0.00 sec) 


mysql> INSERT INTO alias maps (account, alias) 
-> VALUES ('root@kernel-panic.it', 
'root@localhost.kernel-panic.it') ; 


Query OK, 1 row affected (0.00 sec) 
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° p5-Net-Daemon-x.xx.tgz 
® p5-P1LRPC-x.xxxx.tgz 

°  p5-DBI-x.xx.tgz 

°  p5-DBD-mysql-x.xxxx.tgz 


mysql-server-x.x.xx.tgz 


After the installation, you will find various sample 
configuration files in the /usr/locai/share/mysql directory; 
choose the most suitable to your needs and copy it to / 
etc/my.cnf. E.g.: 


# cp /usr/local/share/mysgl/my-small.cnf /etc/my.cnf 


The socket dilemma 

Choosing a good location for the MySQL socket file is 
sometimes hard because of chrooted processes, which 
need to access it from inside their reduced filesystem. But 
Postfix goes even further: of its many processes, most 
are chrooted to the /var/spool/postfix directory, but a few 
are not! As a consequence, by default, part of the Postfix 
processes will look for the socket file in the /var/run/mysql/ 
directory, while the others will look for it in the /var/spoo1/ 
postfix/var/run/mysql/ directory! 

Anyway, there are many possible workarounds: 


e if the database runs on a remote server, there is 
no need to bother with the socket file! We will later 
see how to configure Postfix and Courier-IMAP for 
connecting to a remote database; 

¢ if you want to preserve the defaults as much as 
possible, you can create a symbolic link to the socket 
inside the chroot before the database startup: 


# mkdir -p /var/spool/postfix/var/run/mysql/ 
# ln -f /var/run/mysgl/mysgl.sock /var/spool/postfix/var/ 
run/mysql/mysql.sock 


« Remember to add the above commands to the /etc/ 
rc.local(8)  (Attp://www.openbsd.org/cgi-bin/man.cgi 
?query=rc&sektion=8) script to automatically create 
the link at boot time. 

* you can place the socket inside the Postfix chroot (by 
setting the value of the socket variable in the [mysqld] 
section of /etc/my.cnt to the path of the socket, e.g. / 
var/spool/postfix/mysql/mysql.sock), and give Postfix the 
possibility to choose between two distinct paths: / 

non-chrooted 

chrooted 


var/spool/postfix/mysql/mysql.sock, for 
processes, and 
processes; 

* finally, you can forget about socket files and connect 
through the loopback network interface. 


/mysql/mysql.sock, for 


www.bsdmag.org 








Listing 8. Additional Postfix configuration for MySQL 


/etc/postfix/mysql virtual_domains.cf 
user = vmail 


password = vmail 


# solution 1: 

# hosts = db server name 

# Solution 2: skip this parameter 

# Solution 3 (this file is required only by chrooted 
processes): 

# hosts = unix:/mysql/mysql.sock 

# Solution 4: 

Hostse— wei MOP Oca 


dbname = mail 

query = SELECT domain FROM domains WHERE domain='%s' 
/etc/postfix/mysql virtual alias _maps.cf 

user = vmail 


password = vmail 


# solution 1: 

# hosts = db server name 

# Solution 2: skip this parameter 

# Solution 3 (this file is required only by chrooted 
processes); 

# hosts = unix:/mysql/mysql.sock 

# Solution 4: 

HOSS eNO Oat 


dbname = mail 

query = SELECT alias FROM alias maps WHERE 
account='%s' 

/etc/postfix/mysql_ virtual_mailboxes.cf 

user = vmail 


password = vmail 


# solution 1: 

# hosts = db server name 

# Solution 2: skip this parameter 

# Solution 3 (this file is required by both chrooted 
and non-chrooted processes) : 

# hosts = unix:/mysql/mysql.sock unix:/var/spool/ 
postfix/mysql/mysql.sock 

# Solution 4: 

Hosts elAi Ole. 


dbname = mail 


query = SELECT maildir FROM users WHERE login='%s' 
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Listing 9. Configuring MySQL-based authentication for Courier 


/etc/courier/authmysqlre 


MYSQL SERVER LATS 5 Osa 
MYSQL USERNAME vmail 
MYSQL PASSWORD vmail 


# If you connect through the socket: 


#MYSQL SOCKET /path/to/mysql.sock 


#MYSQL PORT 0 

MYSQL PORT 3306 
MYSQL OPT 0 
MYSQL_DATABASE mail 
MYSQL USER TABLE users 
MYSQL CRYPT PWFIELD password 


MYSQL DEFAULT DOMAIN kernel-panic.it 





MYSQL UID FIELD uid 
MYSQL GID FIELD gid 
MYSQL LOGIN FIELD login 
MYSQL HOME FIELD home 
MYSQL NAME FIELD name 
MYSQL MAILDIR FIELD maildir 
MYSQL QUOTA FIELD quota 


# MYSQL WHERE CLAUSE field=value AND field=value... 











Mmmh... what to choose? After a moment's thought, 
| chose the latter solution, which is probably the 
simplest. Therefore, | left sxip networking Commented 
out in /etc/my.cnt and added the following line in the 
[mysqia] section : 


/etc/my.cnf 
bind-address = 127.0.0.1 


thus preventing MySQL from listening on the external 
network interfaces. 


Configuration 

First and foremost, we need to install the default 
databases, change the password of the MySQL root 
user (don't take my passwords as an example!): see 
Listing 6, and configure the system to start MySQL on 
boot: 


/etc/rc.local 
if [ -x /usr/local/bin/mysqld_ safe ]; then 

echo -n ' MySQL' 

/usr/local/bin/mysqld_safe >/dev/null 2>81 & 
fi 
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Next, we will hook Postfix up to the database. In 
particular, we will modify the value of a few parameters 
in the /etc/postfix/main.ce file: 


/etc/postfix/main.cf 

virtual_mailbox domains = mysql:/etc/postfix/mysql virtual_ 
domains.cf 

virtual mailbox maps = mysql:/etc/postfix/mysql virtual_ 
mailboxes.cf 

virtual alias maps = mysql:/etc/postfix/mysql virtual_ 


alias _maps.cf 


We will see in a moment the contents of those files; 
but first, we are going to create the database. Tables 
don't need to have any particular structure, since we 
will tell Postfix which queries to use to extract the data. 
Therefore, this will actually be just one among the many 





Listing 10. Testing IMAP functionality 


IMAP test.py 
#!/usr/bin/env python 


import imaplib 


# Constants 
IMAP SRV = "mail.kernel-panic.it" 
USER = "d.mazzocchio@kernel-panic.it" 


PASSWD = "danix" 
# Connect to server 
imap_srv = imaplib.IMAP4 (IMAP SRV) 


imap _srv.login(USER, PASSWD) 


# Select the INBOX folder 


imap_srv.select () 


# Retrieve message list 


msg nums = imap _srv.search(None, 'ALL') [1] 
# Print all messages 
for num in msg _nums[0].split(): 
msg = imap srv.fetch(num, '(RFC822)') [1] 


9 


print 'Message %s\n%s\n' % (num, msg[0] [1] 
# Disconnect from server 
imap _srv.close() 


imap_srv. logout () 
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possible implementations: feel free to modify it according 
to your taste and needs. 

Note: Postfix obtains the full pathname of the maildirs 
by joining the values of the virtual mailbox base and 
virtual_mailbox maps parameters, while Courier-IMAP 
obtains it by joining the values of the myso. some FreLp 
and myson matnprr FIELD parameters. As a consequence, 
we will create two separate fields in the users table (nome 
and maiiair) and make those variables point to them in 
order for Postfix and Courier-IMAP to get along see 
Listing 7. 

Now let's take a brief look at the new Postfix configuration 
files, which include the configuration settings for MySQL 
see Listing 8. 

That's all: now we can reload Postfix configuration: 


# postfix reload 


postfix/postfix-script: refreshing the Postfix mail system 


and test our work; everything should run exactly as 
before! 


Courier-IMAP 

Now that our server can send and receive email, it 
may be useful to let users read it! For this purpose, 
we're going to install Courier-IMAP (http:/;vww.courier- 
mta.org/imap/), a fast, scalable, enterprise IMAP server 
that uses Maildirs. This is the same IMAP server that 
comes with the Courier mail server (http:/;vww.courier- 
mta.org/), but configured as a standalone IMAP server 
that can be used with other mail servers, such as 
Postfix. 


Installation and configuration 
The following is the list of the required packages: 


° gdbm-x.x.x.tgz 

° libltdl-x.x.x.tgz 
tcl-x.x.x.tgz 
expect-x.x.x-no_ tk.tgz 
courier-authlib-x.x.tgz 
courier-imap-x.x.x.tgz 


courier-authlib-mysql-x.x.x.tgz 


Once you have added all the packages, you will find 
a fresh new /etc/courier/ directory containing Courier 
IMAP's configuration files. Let's take a brief look at each 
of them. 

The /etc/courier/authdaemonre configuration file sets 
several operational parameters for the 
process (the resident authentication 
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fortunately, we only need to edit the authmodulelist 
parameter, which specifies the list of the authentication 
modules available; set it to authmysqi to allow for MySQL 
based authentication: 


/etc/courier/authdaemonre 
L aa J 
authmodulelist="authmysql" 
[vena 4 


The /etc/courier/authmysqirc Configuration file contains 
the authmysql database connection parameters; below is 
a sample configuration file: see Listing 9. 

The next step is creating the SSL certificate for the 
IMAPS protocol. To make your life easier, Courier-IMAP 
comes with a script, mkimapdcert(s) (Attp://www.courier- 
mta.org/mkimapdcert.html), which will create the 
certificate after reading all the necessary information from 
the /etc/courier/imapd.cnf configuration file. Therefore, 





Listing 11. Testing the POP service 


# telnet mail.kernel-panic.it 110 
Trying) diZdlioe24 0 USO) 

Connected to mail.kernel-panic.it. 
scape character ass |/or 

+OK Hello there. 

user d.mazzocchio@kernel-panic.it 
+OK Password required. 

pass danix 

+OK logged in. 

aeSite 

2531 


quit 
+OK Bye-bye. 





Connection closed by foreign host. 
# 


Listing 12. Freshclam configuration 


/etc/freshclam.conf 


DatabaseDirectory /var/db/clamav 
DatabaseOwner _clamav 
DNSDatabaseInfo current.cvd.clamav.net 
DatabaseMirror db.it.clamav.net 
DatabaseMirror database.clamav.net 
MaxAttempts 2 

checks 24 
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you should first customize the latter file (in particular, pay 
close attention to the common name (CN) parameter, 
which must match the name of the server the users will 
connect to) and then run mkimapdcert (8) (Attp://www.courier- 
mta.org/mkimapdcert.html): 


# /usr/local/sbin/mkimapdcert 
[eee 


Now we only have to start the daemons: 


# mkdir -p /var/run/courier{,-auth}/ 
# /usr/local/sbin/authdaemond start 
# /usr/local/libexec/imapd.re start 


# /usr/local/libexec/imapd-ssl.rc start 
configure the system to start Courier-IMAP on boot: 


/etc/rc.local 

echo -n ' Courier-IMAP' 

/bin/mkdir -p /var/run/courier{,-auth}/ 
[ -x /usr/local/sbin/authdaemond ] && /usr/local/sbin/ 


authdaemond start 





Listing 13. Updating virus signatures with freshclam 


# freshclam 

ClamAV update process started at Tue Dec 18 00:35:25 2007 
WARNING: Your ClamAV installation is OUTDATED! 

WARNING: Local version: 0.90.3 Recommended version: 0.92 
DON'T PANIC! Read http://www.clamav.net/support/faq 
Downloading main.cvd [100%] 
main.cvd updated (version: 45, sigs: 169676, f-level: 
21, builder: sven) 

WARNING: Your ClamAV installation is OUTDATED! 
WARNING: Current functionality level = 16, recommended 
= 21 

DON'T PANIC! Read http://www.clamav.net/support/faq 
Downloading daily.cvd [100%] 
daily.cvd updated (version: 5160, sigs: 8698, f-level: 
21, builder: sven) 

WARNING: Your ClamAV installation is OUTDATED! 


WARNING: Current functionality level = 16, recommended = 21 





DON'T PANIC! Read http://www.clamav.net/support/faq 
Database updated (178374 signatures) from 
db.1t.clamavy.net {1P: 
IG 2065132). 37) 
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[ -x /usr/local/libexec/imapd.rc ] && /usr/local/libexec/ 
imapd.re start 
[ -x /usr/local/libexec/imapd-ssl.rc ] && /usr/local/ 


libexec/imapd-ssl.rce start 


..and test our hard work! | suggest using a simple 
Python script, just to give our weary fingers a break: see 
Listing 10. 


Adding POP3 access 
It is usually desirable that email users be offered the 
choice between IMAP and POP3 remote access; after all, 
POP3 users tend to use less disk space, bandwidth and 
resources on the server. 

Adding POP3 support to our mail server is fairly simple; 
first, we need to add the appropriate package: 


courier-pop3-x.x.x.tgz 


Then, we have to run mkpop3dcert(s) (Attp:// 
www.courier-mta.org/mkpop3dcert.html) to generate 
the SSL certificate for POP3 over SSL (similarly 
to mkimapdcert (8) (http://www.courier-mta.org/ 
mkimapdcert.html), SSL parameters are read from 
a configuration file, /etc/courier/pop3d.cnt) and start the 
daemons: 


# /usr/local/sbin/mkpop3dcert 

[tee 4 

# /usr/local/libexec/pop3d.re start 

# /usr/local/libexec/pop3d-ssl.rce start 


Add the following lines to /etc/rc.1ocai(s) (Attp:// 
www.openbsd.org/cgi-bin/man.cgi?query=rc.local&sekti 
on=8) to start the POP3 server on boot: 


/etc/rc.local 

[ -x /usr/local/libexec/pop3d.rce ] && /usr/local/libexec/ 
pop3d.re start 

[ -x /usr/local/libexec/pop3d-ssl.rc ] && /usr/local/ 


libexec/pop3d-ssl.rc start 


Finally, we can perform a quick test to make sure 
everything works as expected: see Listing 11. 


Managing disk space 

Quotas allow you to specify the maximum size of maildirs, 
in order to prevent the /var/vmaii filesystem from filling up. 
The best option would certainly be to use the operating 
system's built-in quota support (http:/Avww.openbsd.org/ 
faq/faq10.htmi#Quotas), but we can't, because we have 
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a single user writing to all the maildirs. Therefore, we 
must rely on the mail software to get quota support on 
maildirs. 

Courier-IMAP comes with built-in quota support, but this 
solves only one half of the problem: in fact, also Postfix 
must be able to reject mail sent to over-quota users. 
To achieve this, we will rely on the detiverquota(s) (http: 
//www.courier-mta.org/deliverquota.htm!) utility, © which 
delivers mail taking into account any software-imposed 
quota on maildirs. 

The first step is assigning a quota to each maildir 
with maildirmake (1) (http://www.courier-mta.org/ 
maildirmake.html). E.g.: 


# /usr/local/bin/maildirmake -q 100000008 \ 


> /var/vmail/kernel-panic.it/d.mazzocchio 


The above command installs an (approximately) 10MB 
quota on_ the 
maildir. Note: maiidirmake(1) (Attp://www.courier-mta.org/ 
maildirmake.html) also allows you to create and initialize 
maildirs, thus allowing users to access them; otherwise, 
a user's maildir will be created upon receiving his first 
email. 

Next, we need to define, in /etc/postfix/master.cf (5) 
(http://www. postfix.org/master.5.html), a special Postfix 


/var/vmail/kernel-panic.it/d.mazzocchio 


daemon for delivery through aeliverquota(s) (http:// 
www. courier-mta.org/deliverquota.htm): 
/etc/postfix/master.cf 
Ib kaa. 
qdeliver unix - n n = = pipe 
flags=uh user=vmail argv=/usr/local/bin/deliverquota -c 
-w 90 


/var/vmail/${domain}/S${user} 


and tell Postfix to use this daemon for final delivery 
to virtual domains, by setting the value of the virtual_ 
transport parameter in /etc/postfix/main.cf: 


/etc/postfix/main.cf 


virtual_transport = qdeliver 


deliverquota(s) Will place a warning message into the 
maildir if, after the message is successfuly delivered, 
the maildir is at least 90 percent full (-w 90). The body of 
the warning message is copied verbatim from the /etc/ 
courier/quotawarnmsg file. 

Please note that, as reported by Giovanni 
Bechis, deliverquota (8) (http://www.courier-mta.org/ 
deliverquota.html) fails to correctly deliver emails sent 


BSD 


MAGAZINE 


34 


to an alias that maps to multiple accounts, one of which 
has the same name as the alias itself, unless you set the 
following parameters IN /etc/postf£ix/main.cf: 


/etc/postfix/main.cf 
qdeliver_destination_concurrency limit = 1 


qdeliver_ destination _recipient_limit = 1 





Listing 14. Amavisd configuration 


/etc/amavisd.conf 
# COMMONLY ADJUSTED SETTINGS: 


$max_servers = 2; 


$daemon_user = ' clamav'; # Run under the same 
user as ClamAV 


$daemon_group = ' clamav'; # Run under the same 


group as ClamAV 
Smydomain = 


"kernel-panic.it'; 


SMYHOME = 
STEMPBASE = "SMYHOME/tmp"; 


'/var/amavisd'; 

# working directory, 
needs to be created manually 

SENV{TMPDIR} = STEMPBASE; 


SQUARANTINEDIR = '/var/clamav/quarantine'; 


# Leave only ClamAV uncommented 
@av_scanners = ( 
['ClamAV-clamd', 
\&ask daemon, ["CONTSCAN {}\n", "/var/clamav/ 
clamd.sock"], 
gr/\bOKS/, qr/\bFOUNDS/, 


qr/*.*?: (?!Infected Archive) (.*) FOUNDS/ ], 


# Leave only ClamAV uncommented 
@av_scanners backup = ( 
['ClamAV-clamscan', 'clamscan', 
"--stdout --disable-summary -r --tempdir=$TEMPBASE 
Leer OR rail 


qr/*.*?: (?!Infected Archive) (.*) FOUNDS/ ], 
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Listing 15. Running amavisd 


# mkdir /var/amavisd/tmp 

# chown -R clamav: clamav /var/amavisd/ 

# /usr/local/sbin/amavisd debug 

Dec 18 22:07:11 mail.kernel-panic.it /usr/local/sbin/ 
amavisd[24429]: starting. 

/usr/local/sbin/amavisd at mail.kernel-panic.it 
amavisd-new-2.3.2 (20050629), 
Unicode aware 

Dec 18 22:07:11 mail.kernel-panic.it /usr/local/sbin/ 
amavisd[24429]: user=, 

EUID: 0 (0); group=, EGID: 0 31 20 5 43 2 0 (0 31 20 
By A Si2 0) 

Dec 18 22:07:11 mail.kernel-panic.it /usr/local/ 
sbin/amavisd[24429]: Perl version 
5.008008 


Listing 16. Adding the content-filtering services to Postfix 


/etc/postfix/master.cf 

smtp-amavis unix — im is = 2 smtp 
-o smtp_data_ done timeout=1200 
-o smtp_send_xforward_command=yes 
-o disable dns_lookups=yes 


-O max_use=20 


127.0.0.1:10025 inet n - = = - smtpd 
=O conzener il ter— 
=0) localiirecipirent imaps— 
-o relay recipient_maps= 
[OO smepderestm crioniclasses— 
-o smtpd delay reject=no 
-o smtpd _client_restrictions=permit_mynetworks, reject 
-o smtpd helo restrictions= 
-o smtpd sender restrictions= 
-o smtpd recipient _restrictions=permit_mynetworks, reject 
-o mynetworks style=host 
-o mynetworks=127.0.0.0/8 
-O0 strict rfc821 envelopes=yes 
-o smtpd error sleep time=0 
-o smtpd _soft_ error limit=1001 
-o smtpd hard error limit=1000 
-o smtpd client connection _count_limit=0 
-o smtpd client connection rate limit=0 
-o receive override options=no header body_ 


checks,no unknown recipient checks 
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Setting these parameters to 1 disables parallel deliveries 
to the same recipient. 


Content filtering 

Now we have a fully-functional mail server, able to send 
and receive email and providing remote access to users' 
mailboxes. However, if we don't want our server to 
become an immune carrier of computer viruses or to be 
drowned under a sea of spam, we need to install all the 
necessary content-filtering tools. 

Though Postfix natively supports multiple content 
inspection mechanisms (http://www. postfix.org/ 
BUILTIN_FILTER_README.html), the documentation 
(http://www. postfix.org/CONTENT_INSPECTION __ 
README.html) itself encourages the use of external 
filters and standard protocols because this allows you 
to choose the best MTA and the best content inspection 
software for your purpose. Therefore, we will rely on 
third-party software for content filtering; in particular, we 
will use SpamAssassin to filter soam, ClamAV to check 
emails for viruses and Amavisd-new to coordinate it 
all. Below is the outline of the whole architecture: see 
Figure 2. 


SpamAssassin 

SpamAssassin _ (http://viki.apache.org/spamassassin/) 
is a mature, widely-deployed open source project that 
serves as a mail filter to identify Spam. SpamAssassin 
uses a variety of mechanisms including header and 
text analysis, Bayesian filtering, DNS blocklists, and 
collaborative filtering databases. 

There are quite a few packages we need to install: 


*° p5-Compress-Raw-Zlib-x.x.tgz 
° p5-10-Compress-Base-x.x.tgz 
° p5-I10-Compress-Zlib-x.x.tgz 
° p5-Compress-Zlib-x.x.tgz 

° p5-I0-Zlib-x.x.tgz 

° p5-10-String-x.x.tgz 

° p5-Algorithm-Diff-x.x.tgz 

° p5-Text-Diff-x.x.tgz 

° p5-Archive-Tar-x.x.tgz 
re2c-x.x.x.tgz 

°  p5-Net-CIDR-Lite-x.x.tgz 

©  p5-Net-IP-x.x.tgz 

° p5-Digest-SHA1-x.x.tgz 

* p5-Digest-HMAC-x.x.tgz 

°  p5-Net-DNS-x.x.tgz 

° p5-Sys-Hostname-Long-x.x.tgz 
°  p5-URI-x.x.tgz 


° p5-Mail-SPF-Query-x.x.tgz 


07/2010 


OpenBSD as a Mail Server 


°  p5-Socket6-x.x.tgz 

° p5-IO-INET6-x.x.tgz 
bzip2-x.x.x.tgz 
libiconv-x.x.x.tgz 

° gettext-x.x.x.tgz 
libidn-x.x.x.tgz 
curl-x.x.x.tgz 
gnupg-x.x.x.tgz 

° p5-Net-SSLeay-x.x.tgz 

° p5-I0-Socket-SSL-x.x.tgz 
° p5-HTML-Tagset-x.x.tgz 
° p5-HTML-Parser-x.x.tgz 
° p5-Crypt-SSLeay-x.x.tgz 
° libghttp-x.x.x.tgz 

© p5-HTTP-GHTTP-x.x.tgz 
p5-libwww-x.x.tgz 


° p5-Mail-SpamAssassin-x.x.x.tgz 


After the packages installation, you will find the main 
SpamAssassin configuration file (ocai.ce) in the fresh 
NeW /etc/mail/spamassassin directory. The configuration 
phase can be very complex and goes beyond the scope 
of this document; anyway, you can find all the details 
in the man page (Mail::SpamAssassin::Conf http:// 
Spamassassin.apache.org/full/3.1.x/dist/doc/Mail_ 
SpamAssassin_Conf.html). 

Like Postfix, SpamAssassin has a lot of configuration 
parameters, although, in most cases, default values can 
be preserved and only a few parameters need to be 
overridden: 


/etc/mail/spamassassin/local.cf 
rewrite header Subject ***** SPAM ***** 
report _safe 1 

lock_method flock 


required score 8.0 


/var/clarmav/clarnd.sock 


Amavisd-new 


127,.0,0,1:10024 


127,.0,0,1:10025 Sp 
SK] [rest -— 


Figure 2. Mail filtering 





www.bsdmag.org 


ClamAV 

ClamAV (http:/Awww.clamav.net/) is an open source 
(GPL) anti-virus toolkit for UNIX; the main purpose of 
this software is the integration with mail servers (i.e. 
attachment scanning). All the antivirus tasks are handled 
by three processes: 


¢ freshclam — which automatically updates the virus 
definitions, by connecting to one of the ClamAV 
mirrors = (http:/Avww.clamav.net/mirrors.html); __ its 
configuration file is /etc/freshclam.confj 

* clamd — a flexible and scalable multi-threaded 
antivirus daemon; its configuration file is 
clamd.conf, 


* clamscan —a command line antivirus scanner. 


/etc/ 


The required packages are: 


arc-x.xx.tgz 
Lha-x.xx.xXXxXxXxx.tgz 
unzip-x.x.tgz 
ZOO-X.X.x.tgz 


gmp-x.x.x.tgz 


unarj-x.x (from the ports) 


unrar-x.x (from the ports) 


clamav-x.x.tgz 


The freshclam.cont Configuration file requires only a few 
parameters: see Listing 12. 

Now we can update the virus definition database by 
running the freshclam Command. Please make sure you 
have installed the latest release of ClamAV, or you'll get 
warning messages about reduced functionality, like the 
following: see Listing 13. 

The reduced functionality level means that you may 
not be able to use all the available virus signatures 
and, consequently, fail to detect the latest viruses. To 
automatically update the database, we simply have to 
schedule freshclam in crontab every hour (preferably not 
on the hour, just to avoid traffic peaks): 


16 * * * * /usr/local/bin/freshclam >/dev/null 2>é1 


Also the /etc/clamd.cont configuration file needs editing 
only very few parameters: 


/etc/clamd.conf 


DatabaseDirectory /var/db/clamav 
LocalSocket /var/clamav/clamd.sock 
User _clamav 
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+ — http://www.postfix.org/BASIC_CONFIGURATION_README.html — Postfix Basic Configuration 

+ — http://www.postfix.org/MYSQL_README.html - Postfix MySQL Howto 
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+ — http://spamassassin.apache.org/full/3.0.x/dist/doc/Mail_SpamAssassin_Conf.html - Mail::SpamAssassin::Conf 

+ — http://www.ijs.si/software/amavisd/README. postfix — How to use amavisd-new with Postfix 

+ — http://www.clamav.net/doc/latest/html/- Clam AntiVirus 0.92 User Manual 

+ — http://www.flakshack.com/anti-spam/wiki/index.php - Fairly-Secure Anti-SPAM Gateway Using OpenBSD, Postfix, Amavisd-new, 


SpamAssassin, Razor and DCC 





Now we can run clamd: 


# touch /var/log/clamd.log 

# chown clamav /var/log/clamd.log 

# clamd 

Running as user clamav (UID 539, GID 539 


and add the following lines to /etc/rc.tocai(s) (http:// 
www.openbsd.org/cgi-bin/man.cgi?query=rc&sektion=8) 
to start it on system boot: 


/etc/re. local 

if [ -x /usr/local/sbin/clamd ]; then 
echo -n ' clamd' 
[ -S /var/clamav/clamd.sock ] && rm -f /var/clamav/clamd.sock 
/usr/local/sbin/clamd >/dev/null 2>é61 

fi 


Amavisd-new 

Amavisd-new — (http:/\vww.ijs.si/software/amavisd/) _ is 
a high-performance interface between mailer (MTA) and 
content checkers. 


References 


We will configure it to bind to port 10024 on the 
loopback interface, where Postfix will forward all 
incoming e-mails. If the e-mail successfully passes 
all the checks, it will be forwarded back to Postfix, 
listening on localhost port 10025; otherwise, mails may 
be deleted or quarantined and the administrator and 
recepients may be notified. 

The following is the list of the required packages: 


° cabextract-x.x.tgz 

° freeze-x.x (from the ports) 
° p5-Convert-BinHex-x.x.tgz 
° p5-I0-stringy-x.x.tgz 

° p5-Mail-Tools-x.x.tgz 

° p5-Time-TimeDate-x.x.tgz 
°  p5-MIME-tools-x.x.tgz 

° p5-Convert-TNEF-x.x.tgz 

° p5-Convert-UULib-x.x.tgz 
° rpm2cpio-x.x.tgz 

° p5-Net-Server-x.x.tgz 

° p5-Unix-Syslog-x.x.tgz 


° amavisd-new-x.x.x.tgz 


« — http://www.kernel-panic.it/openbsd/carp/index.html - Redundant firewalls with OpenBSD, CARP and pfsync 

+ — http:/vww.openbsd.org/- OpenBSD, a FREE, multi-platform 4.4BSD-based UNIX-like operating system 

« — http://vww.postfix.org/— Postfix, an Open source email server for Unix 

+ — http://www.postfix.org/TLS_README.html - [TLS] - Postfix TLS Support 

+ — http:/vww.mysql.com/- MySQL, the world's most popular open source database 

+ — http://vww.courier-mta.org/imap/- Courier-IMAP, a fast, scalable, enterprise IMAP server that uses Maildirs 

* — http://asg.web.cmu.edu/sasl/ -— Cyrus SASL, the Cyrus SASL API implentation 

* — http://www.ijs.si/software/amavisd/ - Amavisd-new, a high-performance interface between mailer (MTA) and content checkers: vi- 


rus scanners and/or SpamAssassin 


+ http://spamassassin.apache.org/ - SpamAssassin, a mail filter to identify spam using a wide range of heuristic tests on mail headers 


and body text 


+ — http://vww.clamav.net/— ClamAV, an open source (GPL) anti-virus toolkit for Unix 

« — http://www.postfix.org/VIRTUAL_README.html - Postfix Virtual Domain Hosting Howto 

+ — http://tools.ietf.org/html/rfc4954 — [RFC4954] — RFC 4954, SMTP Service Extension for Authentication 
http://www.postfix.org/SASL_README.html - [SASL] - Postfix SASL Howto 
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The installation procedure creates a new user and 
group called however, the easiest way to get 
Amavisd-new to cooperate with ClamAV, is to run them 
both under the same user (__ clamav). The configuration 
file is /etc/amavisd.conf, which is actually a perl script (so 
pay attention to the semi-colons at the end of the lines!); 
below are the options you will most likely want to tweak: 
see Listing 14. 

After manually creating Amavisd-new's working 
directory (/var/amavisd/tmp), we can start the daemon in 
debug mode (i.e. in foreground), just to check any errors: 
see Listing 15. 

Now we can configure the system to start Amavisd-new 
on boot: 


_vscan; 


/etc/rc.local 
if [ -x /usr/local/sbin/amavisd ]; then 
echo -n ' amavisd' 
/usr/local/sbin/amavisd >/dev/null 2>é61 


fi 


The last step is to update Postfix configuration to 
enable interfacing between Postfix and Amavisd-new. 
To achieve this, we have to add a couple of services 
to the /etc/posttix/master.ct (5) (Attp://www.postfix.org/ 
master.5.html) configuration file: one to forward all 
incoming emails to Amavid-new, and the other to get 
emails back again:see Listing 16. 

Finally, we need to tell Postfix to start forwarding all the 
emails it receives to amavisd-new for content inspection 
and reload the configuration. 

# postconf -e 'content_filter=smtp-amavis: [127.0.0.1]: 
10024" 
# postfix reload 


postfix/postfix-script: refreshing the Postfix mail system 


Appendix 
Special thanks to TomazZ for his detailed notes on configuring 
TLS in Postfix. 
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The Power to Manage Data 





Performance Comparison 


ITTIA DB and SQLite 


ITTIA DB SQL and SQLite are used by software developers to 
manage information stored in applications and devices. Designed 
to be hidden from the end-user, these embedded relational 
database management systems are linked into the application or 
firmware as self-contained software libraries. 


his greatly simplifies the application code 
| responsible for organizing data and sharing it 
between concurrent tasks, while protecting data 

from corruption and race conditions. 

SQLite is an open source software library that 
provides the basic features for storing information in an 
SQL database. SQLite is designed for self-contained 
applications that require an SQL database, but do not 
frequently modify or share access to the database. 

ITTIA DB SQL is a scalable database engine 
that supports a wide range of features, providing 
high performance for self-contained applications 
without limiting the application's ability to scale up as 
requirements change. ITTIA DB SQL can be used as 
either a stand-alone software library, or with a lightweight 
server. 

This white paper explores how differences between ITTIA 
DB SQL and SQLite affect the performance, features, and 
maintenance of database-driven applications. 


Criteria for Selecting a Lightweight Relational 
Embedded Database 

Every software development project has unique 
requirements, including performance objectives, 
reliability expectations, and time-to-market constraints. 
Careful planning, and the incorporation of the right 
database technology, can dramatically reduce project 
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development time while increasing the performance and 
reliability of an application. 

Databases can be implemented in many different 
ways, and the choice of algorithms in a_ particular 
database product has a profound impact on performance 
characteristics and what features are available. Areas 
most impacted by the implementation of the database 
include: 

e Frequency of costly disk or flash media 1/O 
operations. 

¢ Time required to recover from a crash or power loss. 

¢ Ability to manage large amounts of data without 
severe performance degradation. 

¢ Performance impact of sharing data between tasks 
and other applications. 

¢ Relative performance of read and write operations. 

¢ Portability of the storage format and application code. 

¢ Effort required to integrate database technology into 
the application. 


Database software must carefully balance the flow 
of data between persistent storage and memory. If 
information is not saved regularly, recovery from a power 
loss will be slower. If information is saved too often, 
performance will suffer and some media, such as flash 
memory, will wear out more quickly. 
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Performance Comparison - ITTIA DB and SQLite 


Table 1. Benchmark Hardware 


256 MiB 


400Mhz 64 MiB 


Windows XP PC_ Windows XP 1.7GHz 


Linux Device Angstrom Linux ARM9 


Scalability is a concern for many applications, both 
for the amount of data that can be stored and for the 
impact of sharing data. If scalability is not important, 
a database can greatly optimize performance. If 
scalability is needed, or may be needed in the future, the 
performance cost can be very high if the database has 
not provisioned for it. 

By considering the current and future requirements of 
an application, an embedded software developer can 
select the most appropriate database technology. 


Benchmark 
This benchmark will measure elapsed time for three 
operations: 


e Insert rows into an empty database. 
¢ Select rows using indexed search. 
¢ Update rows using indexed search. 


Test Environment 
The benchmark is performed on the following platform: 
see Table 1. 


Test Methodology 


Product Configuration 
For this benchmark, the database products are configured 
as follows: see Table 2. 


Database Schema 
The following database schema is used for the benchmark: 
see Listing 1. 


Table 2. xxxxxxx 


Parameters 
Each product is tested for both disk- 
based and_ in-memory _ storage. 


Additionally, ITTIA DB is tested using 
both table cursors and SQL queries. 
SQLite only supports SQL queries. The 
benchmark is run with the following 
parameters for disk-based storage: 


Hard Disk 


Flash 


¢ inserts = 10,000 

« selects = 10,000 

¢ updates = 10,000 

* max_inserts_per_tx = 40 

* max_selects_per_tx = 120 
* max_updates_per_tx = 80 


The number of operations is increased for in-memory 
storage: 


* inserts = 100,000 
« selects = 100,000 
* updates = 100,000 


Test Algorithms 

SQL queries are prepared once at the beginning of the 
test. When table cursors are used, SQL queries are 
replaced with equivalent ITTIA DB C API syntax see 
Listing 2 and Listing 3. 


Results 
Results are measured as a ratio of elapsed time for SQLite 
to elapsed time for ITTIA DB. A result of 1.00 indicates 
equivalent performance. Values greater than 1.00 favor 
ITTIA DB. Values less than 1.00 favor SQLite. 

Metrics for elapsed time are available upon request. 
Contacting ITTIA Research and Development using the 
form at: http://www. ittia.com/contact 


¢ Disk Storage see Figure 1 and Figure 2. 
« Memory Storage see Figure 3 and Figure 4 
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Atomic Transactions 


Storage Type ane em In an embedded database, related 


Stand-alone single-user library Stand-alone library 


database operations are grouped 


CS ec coved other comeletaty or notat 


ITTIA DB C API 


SQLite C API 


will be saved either completely or not at 
all. To the application, it is as though all 


SST HaiS GHC SORI ECO changes made in the transaction are 
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when the transaction is committed. This feature is known creates a rollback journal for each transaction that 
as atomic commit. remembers the original value before each change is 

ITTIA DB SQL and SQLite both support atomic commit, made. ITTIA DB stores both the original value and the 
but with different performance characteristics. SQLite new value in a write-ahead log that can be shared by 








Listing 1. Schema Listing 3. Benchmark Select Algorithm 


create table benchmark table ( const int selects; 
a integer not null, const int max_selects_per tx; 
b integer, 
unusedl integer, begin transaction; 


unused2 integer, 
unused3 integer, for (long i = 1; i <= selects; i++) 

unused4 integer, long value = (rand() % inserts) + 1; 
unusedS integer, 
unused6 integer, select b from benchmark table where a = value; 


unused7 integer, 


unused8 integer, Pee (fee cmiaxm oc lOc tom me amis m0) 
unused9 integer, commit transaction; 
unused10 integer, begin transaction; 


unusedll integer, 
unused1l2 integer, 
unused13 integer, 
unusedl4 integer, commit transaction; 
unused15 integer, 
unusedl6 integer, Listing 4. Benchmark Update Algorithm 
unused1l7 integer, 
unused18 integer const int updates; 

)i const int max_updates per tx; 

create unique index ixl on benchmark table (a); 

begin transaction; 


Listing 2. Benchmark Insert Algorithm 


for (long i = 1; i <= updates; i++) 
const int inserts; long value = (rand() % inserts) + 1; 
const int max_inserts per tx; 


update benchmark table set b = value + 8 where a = 


begin transaction; value; 
for(long i = 1; i <= inserts; i++) if (i % max_updates per tx == 0) 
insert into benchmark table(a, b) values(i, i+2); commit transaction; 


begin transaction; 
if (max_inserts_per_tx > 0 && i % max_inserts per_ 
tx == 0) 
COMMIE Eransaction; 


begin transaction; commit transaction; 


commit transaction; 
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genioDATA Multiverse 


Exactly the IT systems you need 


genioDATA hosts every important operating system (NetBSD, FreeBSD, 
MacOS X Server) and many others as well (Linux, Solaris, Windows, 
HPUX) - on real hardware or virtualized. 


All of them run in premier grade data centers (e.g. Level3). 

This is what you get: 

— clustered virtual machines, 

SAN-Storage (clustered or single), 

redundant routing and switching paths, 

assistance or complete management for your system (optional), 
several backup strategies including regional desaster recovery options. 


Ask other providers if they can keep up: 
-— professional UNIX administrators with various certifications 
(RHCE, ACSA, ...), 
- no long-term contracting (very low customer quit rate: no ties involved), 
- individual setups for elaborated requirements, 
reliable platforms for those who do everything on their own, 
- SLAs for up to 99,999 % availability (depending on the setup), 
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Special discount: 
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- write an email to sales@geniodata.de, 
— use our web contact form: www.geniodata.com/contact. 
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multiple transactions. This has some surprising results on 
overall performance. 

When a transaction is committed, the database must 
write enough information to disk to guarantee that 
no changes made in the transaction are lost. Data is 
organized into pages to optimize access to block devices 
such as disks and flash memory. In most cases, changes 
are scattered throughout the database, only modifying 
a small portion of each page. 

When SQLite commits a transaction, it must write all 
modified pages to disk in their entirety and then wait 
for the operation to finish. This is extremely costly, 
but necessary to support recovery with only a rollback 
journal. 

When ITTIA DB SQL commits a transaction, only the 
write-ahead log must be written immediately because 
it contains a copy of all the changes made in the 
transaction. Log entries are written sequentially and 
only contain information about changes, so fewer pages 
are written to disk. Other modified pages are written 
to disk later, after accumulating changes from many 
transactions. 

Under high load, ITTIA DB SQL can significantly reduce 
the amount of write activity compared to SQLite, both 
boosting performance and reducing wear on flash storage 
media. 


Shared Access 

Device applications usually perform many different tasks 
on data stored in the database. Some tasks perform best 
when run in parallel, allowing long-running tasks such as 
synchronization to happen without completely stopping 
normal operations. Tasks can be performed by a single 
application with multiple threads or instances, multiple 
applications on a device, or even remote applications 
across a network. 

SQLite uses the locking mechanism built-in to the file 
system to protect database files during modification, 
which allows any process on the device with access 
to the file to use the database. This approach relies 
on the stability of the file system locking framework, 
and therefore cannot be used with files that are shared 
over a network. Many operating systems do not fully 
implement the range locks required to safely share an 
SQLite database. 

ITTIADB SQL uses a lightweight data server to negotiate 
shared access. The server is self-contained, requiring little 
or no configuration so that it is straightforward to use on 
a device. To access the database, an application is linked 
with a small client library in place of the stand-alone library. 
No code changes are required to access a database file 
on the same device and the application can also open 
database files remotely over a TCP/IP network. 
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12.00 
10.00 3.00 
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Figure 1. SQLite vs. ITTIA DB Table Cursors (Disk) Figure 3. SQLite vs. ITTIA DB Table Cursors (Memory) 
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Figure 2. SQLite vs. ITTIA DB SQL Queries (Disk) 


BSD 


MAGAZINE 


aa| 


Figure 4. SQLite vs. ITTIA DB SQL Queries (Memory) 
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File system locks operate on an entire file at once, 
so when an SQLite transaction begins to modify the 
database, it must obtain exclusive access to the entire You will find here: 
database file until the transaction is finished. This is 
not a problem when sharing is infrequent and every 
transaction only involves one or a few rows. However, one —~materials for articles- 
long-running transaction, such as a synchronization task, | hd 
can block all other activity in the database, even when listings, additional 
there is no real conflict. - 

ITTIA DB SQL uses a less restrictive locking technique: a ah ah y ba 
row-level locking with isolation levels. The database 


automatically tracks all rows that are read or modified in the most interesting 
a transaction. At the highest level of isolation, known as os 


serializable, rows are locked in such a way as to prevent articles to download 
all possible conflicts. And for most simple transactions, the 

isolation level can be reduced to minimize locking even c i 
further. This ensures that a transaction is only blocked Ys current information 
when it would create a conflict with another transaction on the upcoming 

already in progress. In addition, an entire table can be 2 Cra 

locked manually. et wr? 

Row-level locking is also available when an ITTIA re 
DB SQL database is shared between threads in an > 
application. SQLite allows an open database to be Cc < 
shared between threads, but only one thread can modify He A 
the database at a time. ITTIA DB SQL permits multiple 
threads to concurrently read and modify different rows in Q 
the same database without risk of conflict. 














In-Memory Databases 

Databases are typically designed to store data on 
a block device, such as a hard disk or flash media. Some 
applications have sufficient memory to store data entirely 
in main memory. To optimize for this scenario, both ITTIA 
DB SQL and SQLite support in-memory storage. 

When a database is created in memory, SQLite uses 
its normal paging algorithms to organize the data, but 
does not write any pages to disk. ITTIA DB SQL uses 
algorithms that are specialized for memory tables. In 
this way, ITTIA DB SQL is able to take advantage of 
optimizations such as direct pointers to significantly 
improve performance. 

ITTIA DB SQL also supports hybrid in-memory/on-disk mig 
databases that contain a mixture of memory and disk See 
tables. A similar result can be achieved with SQLite by 
attaching an existing database file to a memory database. r=) 
When tables are stored on disk, ITTIA DB uses row-level 
locking for higher concurrency. yan 








Data Typing a a 
Embedded databases can store many types of at 
information, the most common being numbers, text, date, 
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and time. A value's data type must be known when it is 
read, whether it is used in an expression, displayed as 
text, or accessed natively in a programming language. 
Many types are incompatible: a string of text cannot 
always be used where a number is expected, and large 
numbers will not fit in an 8-bit variable. 

To prevent a type mismatch, the database can 
check a value's data type either when it is written to 
the database, or when it is read. In the first case, the 
database must have some information about how the 
value will be used later so that it can check incoming 
values for conformance, and when a type mismatch 
occurs, the error must be reconciled when the value 
is stored in the database. In the case where the value 
is checked at read time, the database must be able 
to accommodate values of arbitrary type, and type 
mismatch errors are not handled until the value is 
read. 

SQLite uses dynamic run-time typing, which means that 
data types are only checked when a value is read from the 
database and used. This is useful in prototyping, before 
the application's requirements are fully formed, because 
it allows data to be written to the database without much 
regard for how it will be used later. Production code, 
however, must be carefully audited to ensure that type 
mismatches are not possible or can be dealt with in 
a reasonable way. 

ITTIA DB SQL uses static typing, where type 
information is stored in the database schema as part 
of a table's description. Each column can contain 
only a specific type of data. This ensures that type 
mismatch errors are identified early, when there is 
the best chance to successfully fix the mistake. This 
is important when the database is shared between 
applications that are developed separately, as the 
database schema forms a contract by which all parties 
must abide. 


Application Programming Interface 

ITTIA DB SQL and SQLite each have an interactive utility 
that provides access to database files through standard 
SQL commands. While this is suitable for testing and 
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maintenance, applications need a native interface to 
access the database. 

To access data in an SQLite database, applications 
use SQL queries. ITTIA DB SQL similarly supports 
SQL queries, but also provides direct access to 
tables and indexes with low-level table cursors. Table 
cursors have lower overhead than SQL queries and 
allow modifications to be made directly while browsing 
a table, without constructing an update query. In many 
cases, an application can use table cursors to both 
improve performance and minimize data layer source 
code. 

To protect a database from unauthorized access, the 
database file can be encrypted. ITTIA DB SQL provides 
encryption callbacks for an application to plug in any 
desired page-level encryption library. SQLite users must 
purchase the SQLite Encryption Extension (SEE), which 
utilizes password-based AES encryption only. Both 
products decrypt data when it is read from disk and 
encrypt data before it is written to disk. 

Developers of embedded systems and_ intelligent 
devices find it important to protect their database from 
unauthorized access. This significant requirement was 
recognized by the architects of ITTIA DB SQL. ITTIA DB 
SQL provides encryption callbacks that an application 
utilizes to plug in any desired page-level encryption 
library. This important feature is not included in the open 
source version of SQLite. 


Technical Support 

The problems solved by an embedded database are not 
trivial. Even though the database software hides most 
of the details of data management from the application, 
learning to use the database effectively takes some time. 
The problem is magnified on restricted hardware, where 
small configuration details can have a big impact on 
footprint. 

The involvement of a database expert can greatly 
improve the quality of the application by identifying the 
best strategies for design and implementation. Except 
for large enterprise companies, the only way to receive 
support for SQLite is through the open source community. 
While this is sometimes sufficient for small, specific 
questions, the community rarely provides high-level 
guidance and cannot provide confidentiality. 

The commercial-grade support offered by ITTIA 
provides critical assistance from database experts 
throughout all phases of development, from laying 
down the best approach in the initial design, through 
development and testing, and into deployment of the 
application. 
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Information in this document is provided solely to enable system and 
software implementers to use ITTIA products. No express or implied 
copyright license is granted hereunder to design or implement any 
database management system software based on the information in 
this document. 

ITTIA reserves the right to make changes without further notice to 
any products described herein. ITTIA makes no warranty, represen- 
tation or guarantee regarding the suitability of its products for any 
particular purpose, nor does ITTIA assume any liability arising out of 
the application of or use of any product, and specifically disclaims 
any and all liability, including without limitation consequential or in- 
cidental damages. Statistics and parameters provided in ITTIA whi- 
te papers and data sheets can and do vary in different applications 
and actual performance may vary over time. All operating parameters 
must be validated for each customer application by customer's tech- 
nical experts. 





Conclusion 

Often used as a stand-alone database software library, 
ITTIA DB SQL greatly simplifies data management for 
applications on embedded systems and devices, but 
without the complexity of a back-end database server. 

SQLite is suited for simple projects that need basic 
transactional storage using SQL. SQLite takes advantage 
of self-imposed limitations to optimize for the most basic 
use cases. Advanced projects with robust or undefined 
requirements can benefit from ITTIA DB SQL's solid 
framework for formalizing, updating, and sharing data. 
Benchmarks show that even for simple operations, ITTIA 
DB outperforms SQLite. 

Each embedded application has unique requirements 
for data management and storage. Selecting the right 
tools requires a careful problem analysis and a thorough 
understanding of database technology. Using a technology 
with the right features makes a significant impact on the 
performance, maintainability, and extensibility of the 
application. 

For any questions about this white paper or benchmark, 
please contact ITTIA Research and Development using 
the form at: http://www.ittia.com/contact 


PR ITTIA ITTIA and the ITTIA logo are trademarks or registered 
trademarks of ITTIA, L.L.C. in the U.S. and other 

countries. All other product or service names are the property of their 

respective owners. 

Copyright 2008-2010, ITTIA L.L.C. All rights Reserved. 
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LET’S TALK 


Interview with 


Jeff Roberson 


Any administrator who has rushed to bring a system back 
on-line after a crash knows how frustrating it can be to sit 
through a filesystem check. It can be a painfully slow, yet 
necessary process. One BSD developer, Jeff Roberson, has 
found a way to make all our lives easier and system recovery 
faster. Jeff took some time out of his very busy schedule to 
explain some of the bottlenecks in filesystem recovery and 
how he has gone about speeding up the process. 


To start off, could you tell our readers a little 
about yourself. Where you're from and how you 
got involved with BSD? 

JR: I've been writing software professionally since 1997 
and I've been an independent contractor since 2002, 
the same year | became a FreeBSD committer. Before 
BSD | did some hacking on Linux but switched and never 
looked back after taking a job working on a FreeBSD- 
based product. | wrote the ULE scheduler, UMA kernel 
memory allocator, parts of the thr threading support, as 
well as quite a lot of work on SMP scalability. | live on the 
island of Maui with my wife, dog, and cat but I'm originally 
from Virginia. 


Could you please explain the previous 
limitations to the FreeBSD filesystem? What 
was the motivation behind your work? 

JR: FreeBSD's FFS (Berkeley Fast File System) 
implementation has a feature called soft-updates which 
permits filesystem meta-data to be written asynchronously 
without fear of corruption after an unclean shutdown. 
While this feature prevents unsafe filesystem states it can 
potentially leak blocks or inodes in the event of a crash or 
power loss. To recover from this, a background fsck is run 
which finds unclaimed blocks and inodes and frees them. 
This process is relatively expensive and time consuming 
which detracts from the utility of the near instant boot 
provided by the ordering guarantees. 
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You've added a journal to the existing 

file system, could you please explain 

how this will help end-users and system 
administrators? 

JR: With the addition of a small intent log the background 
filesystem check step is eliminated. The fsck program has 
been augmented to parse the journal and recover only 
those operations which were pending at the time of the 
crash. On a large and fully populated filesystem this can 
reduce fsck time from many hours to few seconds. The 
recovery time actually scales with the size of the journal 
and not the size of the filesystem. Bigger filesystems need 
not take longer to recover. The journal size itself is limited 
by how many dirty or uncommitted meta-data changes 
can be in memory at once. 


Will your changes be compatible with existing 
BSD file systems? For instance, will it be 
possible for people to use your journal on 
existing file system without re-formatting? 

JR: The meta-data changes were very minimal. The 
journal itself is just a special file in the filesystem. Enabling 
journaling may be done with tunefs on an unmounted 
filesystem. Journaling may then be disabled also with 
tunefs. There are no significant compatibility issues. 
An older FFS implementation can even fsck and mount 
a journaled filesystem, although care should be taken to 
ensure that it is clean when returning to an implementation 
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that does support journaling as there could otherwise be 
stale recovery information. 


Was it difficult to add a journal to the existing 
filesystem code? Were there moments when 
you considered starting a new FS from 
scratch? 

JR: While | never considered starting a whole new 
filesystem, it was indeed very difficult. This took roughly 
500 hours to get to a state where it could be committed 
to head and used by the general public. This is not 
the simple full-block journaling that many filesystems 
implement. The recovery operation has to parse the 
available logs and the current filesystem state and make 
an determination of what it should be. The journal writes 
in the kernel had to be ordered properly with other meta- 
data writes in soft-updates. The end result was nearly 
10,000 lines of code which favourably compares to 
advanced journaling implementations in other operating 
systems. 


To take advantage of your work, will people 
need to upgrade to FreeBSD-current, or will 
your work be ported to existing stable versions, 
ie FreeBSD 8? 

JR: There are currently backports in the project's SUJ 
FreeBSD svn repository. We are still discussing how long 
these will be maintained and if there will be an official 8.x 
release with SUJ. A lot of that depends on how stable and 
performant the code turns out to be in -current. For now 
| would suggest those that want to try SUJ run current and 
report any bugs or issues that arise on the freebsd-current 
email list. 


You've mentioned on your blog that some 
companies (iXsystems and Yahoo) were 
sponsoring your work. How did that come 
about? 

JR: | had an initial idea of hinting fsck with a journal 
at BSDCan and discussed it with a few people. | was 
considering at the time ways that filesystems are limited 
by sticking to only one meta-data coherency protocol 
(softdep, journaling, copy-on-write). After | was convinced 
that the idea would work | produced a proposal and draft of 
a technical document and began to pursue contributions. 
Ultimately about half of my time was paid for by corporate 
sponsors but the benefit to FreeBSD and the experience 
gained was worth it. 
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If l! understand your latest blog post, you've 
made recovery from a filesystem crash up to 
1,200 times faster. That's a huge improvement. 

| have to wonder why there wasn't a strong 
push to do this sooner. What made you be the 
first? 

JR: | can't say exactly that | was the first. The gjournal 
project by pjd provided full block logging with FFS many 
years prior. There are certain overheads with full block 
logging which meant it didn't see as much use as it could, 
and perhaps should, have. Many were deterred from 
implementing journaling because it is a big task no matter 
how you do it. | think several people attempted to make 
fully journaled versions of FFS but | was probably the 
first to reduce the scope by integrating with soft-updates 
rather than replacing it. Many companies have an interest 
in this type of infrastructure work but lack the expertise or 
man power to ultimately tackle it. 


This was a big task, were you working with 
anyone else? Other FS gurus or people testing 
changes with you? 

JR: | worked closely with the original author of FFS and 
soft-updates, Kirk McKusick. He reviewed my changes, 
and provided design feedback as well as essential 
details about the rationale and design of the existing 
mechanisms. Peter Holm was instrumental in testing. His 
kernel stress suite and new custom tests made this a far 
more stable and higher-quality system than | could've 
done alone. Scott Long, Matt Olander, and Debbie Chu 
should also be thanked for their parts in organizing 
funding for the project, without which | could not have 
done it. Many others have provided excellent bug reports 
and performance feedback. 


Do you have any other projects on the go? 
Other things you'd like to tackle? 

JR: I'm just starting a project to bring Infiniband to 
FreeBSD that will consume some time. One day | would 
love to have the funding and support required to write 
a new filesystem from scratch. Until then | spend my free 
time on other infrastructure projects in FreeBSD. 
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FreeBSD 


Experience and Success Story 


of the Philippines Open University (UPOU). | came 

from a Microsoft Windows platform and Visual Basic 
background. At UPOU, there was no room for for my skills 
since they use various distributions of Linux for servers 
and open source programming languages for applications 
development. 

Being a new employee (that time), | was never part of 
server discussions, since | never knew what they were 
talking about. Fedora, Ubuntu, Debian, NFS, FTP, those 
were the things | heard, but | was all at lost in those 
technical terms. 

After a few months, | decided to try my hands on those 
system so that | may be able to join the discussion and be 
a part of the systems administration meeting and team. 
| tried my hands on a popular Linux distribution using 
KDE. | was quite impressed by KDE, but to be honest, 
| got a lot of system crashes(KDE related) so | decided to 
look for another one. | came across at this website(| can't 
remember) with a little devil logo. | clicked the link and it 
brought me to the freebsd website. 

| downloaded the three (3) installation discs of version 
6.2 for i386. At first | was afraid to install it since, 
the installer was not GUl-based, unlike my previous 
experience with the popular Linux distribution. So this 
was my second experience of a non-Windows operating 
system, and my first experience of installing using a non- 
GUI based installed. 

After a couple of failure installations on my laptop, 
| decided to study the documentation first. | read the 
handbook to keep me going. And finally, after a lot of disc- 
switching in my CD-drive, | was able to do a complete 
installation of FreeBSD 6.2 with KDE as my window 
manager. 

Since then, | used FreeBSD 6.2 for my laptop. To cut the 
story short, | became a novice FreeBSD user and | joined 
the FreeBSD Forum to seek more knowledge and skills 
for systems administration, since that was my dream job 
(not being a programmer). | was able to get a grasp of 
how to setup a DHCP, DNS and an FTP server. | was also 
looking at how the Squid Caching server works through 
the FreeBSD forum. | also bought the The Book of PF by 
P. N. M. Hansteen, to learn about setting up a firewall, port 
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forwarding and NAT. | also downloaded PDF files from the 
BSD Magazine website to learn from the articles. 

A couple of months ago, our primary gateway computer 
crashed, leaving our entire organization without a way to 
connect to the Internet. The whole IT team was required 
to setup a new one at the soonest possible time. Our 
system administrator back then, wasn't around and | was 
the only one with technical know-how to do the job. 

| saw the old Sun Ultra 10 Workstation on the server 
room doing nothing. | decided to use that for the gateway. 
| downloaded FreeBSD 8.0 for Sparc and immediately 
installed it. | quickly downloaded the ports collection and 
installed ISC-DHCP 3.1 server first, then DNSMasq, and 
Squid. | recompiled the kernel with PF, created pf.conf 
and tested the system. To no avail, | was able to setup 
a gateway/firewall, DHCP server, DNS server, and Squid 
proxy server in less than half a working day. 

Our organization was back online once again and 
the management was happy because the setup | made 
includes a proxy server which can block unwanted web 
sites. | was able to document my setup and experience. 
| created a generalized how-to article and submitted it to 
BSD Magazine for review and publication. | was surprised 
that my how-to article was published in the April, 2010 issue 
of the BSD Magazine and I'm very thankful for it. You can 
download a copy of the how-to in BSD Magazine website. 

When our system adminstrator left for another company, 
| took charge of the entire systems and network and 
| became the new system administrator. All the thanks 
to FreeBSD, the members of the Documentation Team, 
active members of the FreeBSD forum, BSD Magazine, 
and the author of the Book of PF. 

| now enjoy my job as a system administrator and 
| continue to read the FreeBSD documentation, BSD 
magazine and read the posts and help others in the 
FreeBSD forum. Once again, thanks. 





JOSHUA EBARVIA 

Joshua Ebarvia is a systems administrator, programmer and 
part-time lecturer at the University of the Philippines Open Uni- 
versity. He enjoys working with different operating systems spe- 
cially the UNIX variants and clones. You can reach him at jo- 
shua.ebarvia@gmail.com 
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Powerful 4U Orion ll Storage Series 


Outstanding Performance 

Excellent Cooling Efficiency 

Up to 24 Processing Cores with Hyper-Threading 
Up to 72TB in 4U, Unparalleled Storage Density 
Up to 432TB in 20U of Rack Space Utilizing 
Optional Orion II JBOD Expansion Units 
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To order today call: 1-800-820-BSDi 
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JBOD Expansion Unit 








Notable features include: 


* Dual Intel® 64-Bit Socket 1366 Six-Core, Quad-Core, or Dual-Core, 
Intel® Xeon® Processor 5600/5500 Series 

* 4U Storage Server Chassis with up to 72 TB storage capacity 

* 36 x 3.5 Hot-Swap SAS/SATA HDDs (24 front side + 12 rear side) 

* 1400 W (1+1) Redundant High Efficiency Power Supply 
(Gold level 93%+ power efficiency) 
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upto 6.4GT/s eae 
* Up to 192GB DDR3 1333/1066/800 MHz ECC Registered 
DIMM/24 GB Unbuffered DIMM 


* 2 (x16) PCI-E 2.0, 4 (x8) PCI-E 2.0 (1 in x16 slot), 1 (x4) PCI-E 
(in x8 slot) 
* Intel® 82576 Dual-port Gigabit Ethernet Controller 


iXsystems Introduces the Orion Il 4U Storage Solution 


The iX-N4236 boasts energy efficient technology and maximum, high density storage capacity, creating a 4U 
powerhouse with superior cooling. 


The Orion II has thirty-six hot-swappable SAS/SATA drive bays, providing 50% more storage density than its predecessor. By 
delivering high-end storage density within a single machine, iXsystems cuts operating costs and reduces energy requirements. 


Storage sizes for the iX-N4236 are customizable, with 250GB, 500GB, 750GB, 1TB, and 2TB hard drives available. For 
environments requiring maximum storage capacity and efficiency, 2TB Enterprise-class drives are available from Western 
Digital®, Seagate®, and Hitachi. These drives feature technologies to prevent vibration damage and increase power savings, 
making them an excellent choice for storage-heavy deployment schemes. 


The Intel® Xeon® Processor 5600 Series (Six/Quad-Core) and Intel® Xeon® Processor 5500 Series (Quad/Dual-Core) 

have a light energy footprint, while creating a perfect environment for intense virtualization, video streaming, and management 
of storage-hungry applications. Energy efficient DDR3 RAM complements the other power saving components while still 
providing 18 slots and up to 192GB of memory overall. 


100% cooling redundancy, efficient airflow, and intelligent chassis design ensure that even under the heaviest of workloads, 
the Orion Il remains at an optimal temperature, while still drawing less power than other servers in its class. With a 1400 W 
Gold Level (93%+ efficient) power supply, the entire system works together to efficiently manage power draw and heat loss. 


For more information or to request a quote, visit: 
http://www. iXsystems.com/Orion2 


Powerful. 
Intelligent. 
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